Methods and apparatuses for hierarchically encoding and decoding a bytestream

ABSTRACT

There may be provided a method of decoding a received set of encoded data representing information that has been compressed, the method comprising: obtaining, a set of attribute indicators from the data set, each indicator of the set of indicators is associated with a subset of the data set; and, decoding a plurality of subsets of the data set, comprising: retrieving decoding parameters for each subset according to the attribute indicator associated with each subset; and, decoding each subset according to the retrieved decoding parameters for each subset.

TECHNICAL FIELD

The present invention relates to methods, apparatuses, computer programsand computer-readable media for encoding and/or decoding a sequence ofdata streams such as a bytestream.

BACKGROUND

When transmitting, or storing, image and video data it is particularlyadvantageous to reduce the size of the data. Techniques for encoding anddecoding such data are varied and well known. Contemporary techniquesprovide a compromise between processing efficiency, data quality anddata compression.

Images are typically represented digitally by representing the coloursof an image by a set of components each having a value. For example, thecolours of an image can be represented using an RGB colour model or theYCbCr colour space where each pixel of an image is represented by threedifferent values.

To compress the data, planes of the image are usually first split intoblocks of data elements, such as blocks of 8×8 pixels, and each blockundergoes a domain transformation. Examples include discrete cosinetransform and wavelet transform. As is well known in the art,transformation coding is used to capture correlation structures in thedata.

The transformed data is then quantized to represent the large set ofvalues using a smaller set of values and then typically undergoes afurther compression step, such as entropy coding. Entropy codingutilises frequently occurring values or sequences of values within adata set in order to reduce the volume of data. For example, an entropycoding technique compresses the digital data by representing frequentlyoccurring patterns with few bits and rarely occurring patterns with manybits.

The efficacy of each step depends on the outcome of the previous step.That is, the transformation and quantisation processes are designed tomake the next step in the process more effective. For example, overallcompression can be made more effective if the transform and quantisationprocesses represent the values of the image with frequently occurringsymbols or groups of symbols so that the entropy coding step is moreeffectual.

The output of the entropy coding operation is thus a stream of data andthe decoding operation is performed in a mirrored manner to the encodingoperation. First the stream of data is decoded to recreate theinformation. To generate a series of blocks, the stream is divided andmapped to a plane of data using an inverse of the process used at theencoder and the blocks are then arranged in their location in the planeaccording to the order in which the blocks were received in the stream.For example, in a typical JPEG algorithm the blocks are arranged in a inleft to right, top to bottom pattern and within each block coefficientsare arranged in a zig-zag or serpentine pattern. The blocks are thende-quantised. The blocks are then domain transformed using, for examplea wavelet or discrete cosine transformation.

Such entropy techniques utilise global statistics for the data toidentify the most likely symbols and encode these symbols efficiently.In order to decode an entropy encoded data set it is a requirement toalso specify the statistics used to encode the data. Such statistics arenormally sent in the form of metadata which is typically incompressible.This results in an overhead in the process which must be considered.Whilst it is possible to further subdivide the data to be encoded intosmaller and smaller sets, to do so results in an increase in themetadata overhead. As such it is important to ensure that the growth inmetadata required to describe the data is not greater than the savingsmade by utilising the enhanced entropy encoding, which results in atradeoff.

Some known codecs, such as AVC, will subdivide a frame of data intoblocks and will vary the size of the blocks and will calculatestatistics for the variable sized blocks. Such statistics may then beused in the encoding process. However, due to the metadata overheadassociated with the statistics, again there exists a tradeoff betweenthe amount of statistics and the cost associated with the provision ofsuch statistics.

There remains difficulty in optimising the decoding process for exampleto take advantage of parallel decoding optimisations or separatedecoding of subsets of the stream. Each block is concatenated with theother blocks and sent as one stream and therefore to accurately installeach transformed block in the correct location in the plane each of theprevious blocks must have been decoded sequentially from the combinedstream—the order of the blocks as they appear in the stream dictates thelocation of the block in the grid.

Similarly, to search for and access each block in a stream to allow forparallel or separate decoding is not possible without decoding theprevious blocks. Imagine some kind of boundary symbol between blocks.Then one wouldn't be able to search for a desired block, but the systemwould be able to search for a block (one can not say which one; one canjust grab a boundary symbol guaranteed to be used for no other reason)and access the block. Additionally, some entropy encoding algorithmswill conflate the blocks such that they can't be separated unless theentire steam is decoded in one entropy decoding operation.Alternatively, if each of the parts of the stream have a variable length(in most coding algorithms) identifying suitable boundaries in the datato enable separating the stream into subsets is difficult withoutcompromising compression, further reinforcing the need for sequentialdecoding.

To implement parallel processing, it has previously been proposed todivide the image data into multiple parts and combine compressedstreams. An alternative approach that has been proposed is to scan thecompressed stream for boundaries in the encoded data or alternativelyinsert markers in the stream with predefined codes to aid the scanningprocess. None of the proposed approaches have been shown to be optimal.

It has been previously proposed to encode data in a hierarchical mannerso as to reduce the overall data size of a signal. In such encodingtechniques, residual data (i.e. the data required to correct low qualityinformation present in a base layer) is used in progressively higherlevels of quality. Such a hierarchical technique is described in WO2013/171173 which proposes a tiered hierarchy of renditions of a signal.In this proposed technique, a base layer of quality represents the imageat a first resolution and subsequent layers in the tiered hierarchy areresidual data or adjustment layers necessary for the decoding side toreconstruct the image at a higher resolution. Techniques are proposed inthis WO 2013/171173 which structure the data in each layer to exploitcorrelation in the residual layers to reduce information entropy bytransforming a block of values into a set of directional components.Each layer in this hierarchical technique, particularly each residuallayer, is often a comparatively sparse data set having many zero valueelements.

The concept of a hierarchical, tiered data structure is also disclosedin earlier filed patent application GB1812407.3 and WO2013/171173. Bothof GB1812407.3 and WO2013/171173 are incorporated by reference.

It has previously been proposed to store sparse matrices usingquadtrees. The techniques build a tree to store the spatial structure ofthe matrix. When considering any possible implementation of the knownformats for reconstructing images using sparse matrices, each requiresintensive memory usage. Each of the known formats that demonstrateefficiency gains require a large amount of data to be stored in memoryto properly reconstruct the locations and values of the data in amatrix.

It remains a goal of industry to reduce the size of image and video datastored or transmitted and to reduce the processing time and memoryutilisation of encoding or decoding sparse data sets in imagereconstruction.

SUMMARY

According to an aspect of the present invention there is provided amethod of decoding a received set of encoded data representinginformation that has been compressed, the method comprising: obtaining,a set of attribute indicators from the data set, each indicator of theset of indicators is associated with a subset of the data set; and,decoding a plurality of subsets of the data set, comprising: retrievingdecoding parameters for each subset according to the attribute indicatorassociated with each subset; and, decoding each subset according to theretrieved decoding parameters for each subset. The invention providesfor subsets of a data set to be grouped together according to commonfeatures of the subsets irrespective of the spatial location of thosesubsets. Thus it becomes possible to reduce the metadata overhead of adata set while maintaining optimal decoding performance provided bydecoding different subsets according to different decoding parameters.The attribute indicator may be considered an index value or key.Preferably the set is received as a stream, and each subset is a portionof that stream. In sum, subsets can be grouped together and assigned avalue but those parameters may be indicated separately in the data set.

The groupings may not necessarily be spatial but by correlation and themetadata may not necessarily be intrinsic to the codec. The groupingsprovide a granularity at a data structure level (referred to as aTessara elsewhere).

Preferably the step of retrieving may comprise retrieving a plurality ofdecoding parameters according to each attribute indicator, the pluralityof decoding parameters corresponding to different functions of adecoding process. In this way multiple decoding parameters can besignalled using only a single attribute indicator.

The plurality of subsets may contain encoded data values and wherein themethod may further comprise: recreating the information that has beencompressed from the data values.

Each indicator of the set of indicators may be associated with arespective subset of the plurality of subsets of the data set. Thus thesubsets can be optimised individually according to an index sentseparately which allows for selective optimisation of the subsets.

A plurality of the indicators may be identical.

The encoded data set may be divided into a hierarchy of subsets. Thusthe attribute indicators may be comprised in a different layer of thehierarchical data structure and configured to correspond to a respectivesubset of a subsequent layer to signal correspondence of the attributeindicators to the subsets without explicitly signalling thecorrespondence and reducing the metadata overhead. The attributeindicators of subsets may contain data values obtained by separatelydecoding an attribute metadata subset of the data set. Thus, a firstdecoding process may be performed prior to identifying the parameters todecode the later data sets. This may be considered a combination ofserial and parallel processing.

In certain embodiments, the method may further comprise: obtaining aninitial attribute indicator associated with an initial subset of thedata set; retrieving initial decoding parameters of the initial subsetaccording to the initial attribute indicator; decoding the initialsubset according to the initial decoding parameters, wherein the initialsubset contains the attribute indicators of the plurality of subsets ofthe data set. This embodiment provides that an initial part of the dataset can be decoded from an initial index value without explicit initialsignalling of parameters for an initial subset. Thus for example aheader may signal the index for a first data structure and that datastructure may be decoded to identify index values corresponding to laterdata structures of the encoded data set. In this, way a hierarchicalprocessing technique may be optimised to reduce metadata overhead.

The decoding parameters may include quantization parameters. Thus,quantization parameters for multiple subsets may be signalled accordingto a group of similar parameters thus reducing the metadata overhead andoptimising quantization of subsets of the data while allowing dynamicoptimisation of the parameters.

The decoding parameters may include entropy decoding probabilitymetadata. The entropy decoding probability metadata may represent acumulative distribution function for a range decoding operation. In thisway the entropy decoding operation performed on the dataset can beoptimised for a group of subsets without those subsets having to bespatially proximal in the dataset or information to be compressed.Selective statistical transmission can be provided while reducing theoverall metadata overhead as the statistics need only to be sent once oralternatively can be retrieved from a store at the decoder andreferenced only by a suitable index.

Techniques of the invention may thus be thought of as an element of aheader, i.e. the attribute indicator if signalled in the header, tellsthe decoder attributes by key and not by data then an index refers to aset of keys in a header.

According to a further aspect there is provided an apparatus fordecoding an received set of encoded data representing information thathas been compressed, comprising a processor configured to carry out themethod of the above aspect.

According to a further aspect there is provided a method of encodinginformation to be compressed into an encoded data set, the methodcomprising: encoding a plurality of subsets of the data set, comprising:retrieving encoding parameters for each subset; and, encoding eachsubset according to the retrieved encoding parameters for each subset;and, generating a set of attribute indicators, each indicator of the setof indicators being associated with a subset of the data set accordingto the retrieved encoding parameters for each subset. The step ofretrieving comprises retrieving a plurality of encoding parameters, theplurality of decoding parameters corresponding to different functions ofan encoding process. The plurality of subsets contain encoded datavalues, such that the information to be compressed can be recreated fromthe data values. Each indicator of the set of indicators is associatedwith a respective subset of the plurality of subsets of the data set. Aplurality of the indicators may be identical. The encoded data set maybe divided into a hierarchy of subsets. The attribute indicators forsubsets containing data values are separately encoded as an attributemetadata subset of the data set. The method may further comprise:encoding an initial subset according to retrieved initial decodingparameters, generating an initial attribute indicator associated with aninitial subset of the data set; wherein the encoded initial subsetcontains the attribute indicators of the plurality of subsets of thedata set. The encoding parameters may include quantization parameters.The encoding parameters may include entropy coding probability metadata.The entropy coding probability metadata may represent a cumulativedistribution function for a range coding operation.

According to a further aspect there may be provided an apparatus forencoding information to be compressed into an encoded data set,comprising a processor configured to carry out the method of the aboveaspect.

According to a further aspect of the invention there is provided amethod of decoding an encoded data set comprising a header and apayload, the payload comprising a plurality of encoded data symbolsrepresenting information that has been compressed. The decoder isconfigured to decode subsets of the payload according to a predetermineddecoding process. The method comprises: decoding the header to derive aset of metadata elements; retrieving from the set of metadata elements,a variation parameter which indicates how the predetermined decodingprocess should be varied; decoding the payload according to a modifieddecoding process based on the variation parameter, such that thedecoding process is optimised for the plurality of encoded data symbols;and, reconstructing the information that has been compressed from thedecoded payload.

Thus a decoding module may obtain a value from a header which indicatesthat the decoding module should adapt its operations based on theindication and/or implement a decoding process using the variationparameter or shortcut. This solution allows a decoder to optimise itscomputational efficiency based on the expected content of the data setfor example. In addition, the overall size of the data set can bereduced or optimised depending on the application of the decodingprocess, i.e. how the process it to be applied to information and thedesired information to be recreated.

In one embodiment, the variation parameter may indicate multiple subsetsof the payload shall be decoded according one or more of the samedecoding parameters. Thus individual decoding parameters may not need tobe signalled for parts of the payload and the decoder can optimiseimplementation as checks or variations may not be necessary.

In a particularly advantageous embodiment, the variation parameter mayindicate quantization is disabled, such that the plurality of encodeddata symbols are not quantized during decoding. In this way, thesolution may provide dynamic lossless encoding of an area of a wholedata set such as an image. This may be beneficial for regions ofinterest. Similarly, with the ability to dynamically disablequantization, parameters may not need to be signalled and thequantization steps not performed, reducing data size and computationalefficiency.

The payload may comprise a plurality of data structures each comprisingmetadata and a plurality of encoded data symbols, wherein the metadatamay indicate features of the data structure, encoded data symbols orboth, and wherein the variation parameter indicates that expectedoptimisations on the metadata of the optimisation process have not beenperformed. Such a solution is particularly advantageous when used incombination with other aspects of the disclosure, for example, metadatamay be processed efficiently to improve computational efficiency withoutcompromising data overhead.

The predetermined decoding process may expect the payload to comprise aplurality of data structures each comprising metadata and a plurality ofencoded data symbols, wherein the metadata may indicate features of thedata structure, encoded data symbols or both, and wherein the variationparameter may indicate the data structures comprise only data symbolsand do not comprise metadata. Where there is no metadata, this mayreduce the overall data size but the decoding process may be varied touse a predetermined approach to properly decode the data and reconstructthe information that has been compressed.

In an example, a decoder may know that a data tier is being decoded, andmay keeps track of tiers in the bytestream, so it may not be given anexplicit indication that a tile contains only data leaves not metadataleaves.

This is particularly advantageous when the metadata are node symbols asdescribed in other aspects of the disclosure such that the orderedquadtree is dense.

The variation parameter may indicate groups of subsets of the payloadshall be decoded according to the same decoding parameters. For example,groups may be versions or any type of common aspects of the subsets.Such a solution facilitates optimisations on the data to enable fasterand more efficient processing according to commonality.

In certain embodiments the variation parameter indicates an expecteddecoding parameter for the payload is signalled explicitly in theencoded data set and is not signalled by reference. Thus a predeterminedor standard parameter can be varied on demand.

Further, the payload may comprise a plurality of subsets, each subsetbeing decoded according to a set of decoding parameters, and wherein thevariation parameter may indicate that an indicator element of theencoded data set that indicates which set of decoding parameters shallbe used for a subset references a list of tuples of a plurality of listsof tuples, each element of the tuple referencing a parameter to be usedfrom a different set of parameters. Thus the decoding process can bevaried according to the types or features of the data to be decoded.Where there is repetition in the information signalled, i.e. parameters,the information that needs to be passed can be reduced by referencing atable or otherwise. Thus, the overall data size is reduced andcomputational efficiency is improved.

Alternatively or additionally, the payload may comprise a plurality ofsubsets, each subset being decoded according to a set of decodingparameters, and wherein the variation parameter indicates that anindicator element of the encoded data set that indicates which set ofdecoding parameters shall be used for a subset is a reference to a setof parameters from a plurality of different sets of parameters. Thus,the decoding process may be informed the parameters are listed in atable rather than having to consult a table to find further pointers toother data sets. This advantageous usage provides optimisation dependingon the number and size of the parameters and their frequency.

As can be seen, aspects of the disclosure thus provide a flexible anddynamic way of signalling optimisations and subsequently modifyingdecoding operations to improve computational speed, efficiency andmemory overhead and also to reduce overall data size. Thus, the generaladvantage provided by the variation parameters is to reduce the amountof data to be encoded/decoded and/or to optimise the execution time atthe decoder, for example by optimising processing of the bytestream.This is performed by modifying decoding operations according to aretrieved variation parameter.

According to a further aspect of the invention there may be provided anapparatus for decoding an encoded data set comprising a header and apayload, comprising a processor configured to carry out the method ofthe above aspect.

According to a further aspect of the invention there may be provided amethod of encoding a data set into an encoded data set comprising aheader and a payload, the payload comprising a plurality of encoded datasymbols representing information to be compressed, wherein a decoder isconfigured to decode subsets of the payload according to a predetermineddecoding process, the method comprising: retrieving a variationparameter which indicates how the predetermined decoding process shouldbe varied; encoding the variation parameter in a set of metadataelements into the header; encoding the information to be compressed intothe payload according to an encoding process based on the variationparameter, such that the decoding process is optimised for the pluralityof encoded data symbols. The variation parameter may indicate multiplesubsets of the payload shall be decoded according one or more of thesame decoding parameters. The variation parameter may indicatequantization is disabled, such that the plurality of encoded datasymbols are not quantized during decoding. The encoded payload maycomprise a plurality of data structures each comprising metadata and aplurality of encoded data symbols, wherein the metadata indicatesfeatures of the data structures, encoded data symbols or both, andwherein the variation parameter indicates that expected optimisations onthe metadata of the optimisation process have not been performed. Thepredetermined encoding process may expect the payload to comprise aplurality of data structures each comprising metadata and a plurality ofencoded data symbols, wherein the metadata may indicate features of thedata structure, encoded data symbols or both, and wherein the variationparameter indicates the encoded data structures comprise only datasymbols and do not comprise metadata. The variation parameter mayindicate groups of subsets of the payload are encoded according to thesame encoding parameters. The variation parameter may indicate anencoding parameter for the payload is signalled explicitly in theencoded data set and is not signalled by reference. The payload maycomprise a plurality of subsets, each subset being encoded according toa set of encoding parameters, and wherein the variation parameterindicates that an indicator element of the encoded data set thatindicates which set of encoding parameters have been used for a subsetreferences a list of tuples of a plurality of lists of tuples, eachelement of the tuple referencing a parameter used from a different setof parameters. The payload may comprise a plurality of subsets, eachsubset being encoded according to a set of encoding parameters, andwherein the variation parameter indicates that an indicator element ofthe encoded data set that indicates which set of encoding parametershave been used for a subset is a reference to a set of parameters from aplurality of different sets of parameters.

According to a further aspect of the invention there may be provided anapparatus for encoding a data set into an encoded data set comprising aheader and a payload, the payload comprising a plurality of encoded datasymbols representing information to be compressed, wherein a decoder isconfigured to decode subsets of the payload according to a predetermineddecoding process, the apparatus comprising a processor configured tocarry out the method of the above aspect.

According to further aspects of the invention there may be providedcomputer readable media which when executed by a processor cause theprocessor to perform any of the methods of the above aspects.

DETAILED DESCRIPTION

Examples of systems and methods in accordance with the invention willnow be described with reference to the accompanying drawings, in which:—

FIG. 1 shows a conceptual diagram of an index used to indicate multipleparameters to decode a data structure;

FIGS. 2A-C show a simplified bytestream for transmitting a hierarchicaldata structure;

FIG. 3 shows the hierarchical data structure and the conceptual indexvalues being associated with a data structure of a data tier;

FIG. 4 shows a diagrammatic illustration of an image plane separatedinto subsets;

FIG. 5 illustrates a root tier mapped to a bytestream;

FIG. 6 illustrates a root tier bytestream portion;

FIG. 7 illustrates a root tier data structure;

FIG. 8 illustrates attribute metadata elements mapped to bytestream;

FIG. 9 illustrates an overview of coupled de-sparsification and decodingmodules;

FIG. 10 illustrates a range decoder schematically;

FIG. 11 is a block diagram showing a system for performing an exampleencoding method;

FIG. 12 is an example of an image to be encoded;

FIG. 13 is a flow chart of the methodology of encoding a frame of videodata;

FIG. 14 is an example of z-order traversal of an image partitioned intotiles;

FIG. 15 is an example image subdivided into tiles and subsequentlyclustered into tilesets;

FIG. 16 is a flow chart of the methodology of adaptively encoding aframe of video data;

FIG. 17 is a flow chart of the methodology of encoding a frame of videodata;

FIG. 18 illustrates a cumulative distribution function;

FIG. 19 illustrates an elementary data structure;

FIG. 20 illustrates a fixed-size header;

FIG. 21 illustrates a first variable-sized header;

FIG. 22 illustrates a second-variable sized header; and,

FIG. 23 illustrates a bytestream structure for a frame.

The present invention provides a technique and apparatus for encoding,and decoding, data, in particular image and video data. In the conceptdescribed herein it is proposed to divide a bytestream or bitstream intoa plurality of subsets each of which are to be decoded separately. It isproposed that parameters to enable or facilitate decoding of thosesubsets are signalled elsewhere in the bytestream. The bytestreamincludes a pointer or indicator associated with each subset that enablesthe decoder to retrieve a set of parameters from elsewhere which shouldbe used for that associated subset. Advantageously, each of the subsetsare grouped according to their common features to facilitate optimalprocessing of those subsets.

Thus it becomes possible to reduce the signalling required to processand decode similar subsets separately. This and other benefits of theproposed technique will be outlined below.

Optionally for these purposes it may not matter what the subsets of thebytestream represent. For example, the subset may represent data valuesor residual values that may be used to recreate an image plane or thesubset may be metadata which describes either another subset in thestream or describes an aspect of the image plane or some other data. Thetechnique allows for any subset to be selectively encoded and decodedusing specific metadata without unduly increasing overall bytestreamsize.

The decoding parameters retrieved according to the indicator may besignalled separately within the bytestream itself or they may beretrieved from a store at the decoder, with the bytestream providing anindication of which parameters should be used from that store. Forexample, the indicator will point to an entry in a table and functionmuch like an index. The indicator may be sent in the header of thebytestream, in a different subset of the bytestream to that to bedecoded and associated with each subset through a predetermined mapping,or optionally together with subset and located primarily in thebytestream for ease of association.

Throughout the present description the terms index, index value,indicator, indicator value, attribute index, attribute indicator andattribute key will be used interchangeably to refer to the referencethat is used to signal to the decoder which set of parameters should beused to facilitate decoding a particular subset. The parameters may bereferred to as decoding parameters or attributes.

Specific decoding parameters will be described throughout the presentdescription however examples of categories of metadata that might berequired for decoding each subset include:

-   -   Statistics metadata used for decoding a particular type of data        structure;    -   Quantization parameters metadata used for decoding that data        structure; Statistics metadata used for decoding a metadata part        of a data structure in conjunction with a data part of a data        structure which uses different statistics; and,    -   Auxiliary attribute metadata used with a data structure, for        example to signal to the decoder that the data structure        represents a particular feature such as a car or sky or any        other auxiliary or auxiliary metadata to selectively signed        information from the encoder to the decoder for a group of        subsets.

In each indicator there may be multiple index values, each index valueof the indicator referring to a single parameter to be used. However,preferably, each indicator will refer to a set of parameters each havinga different function or type. That is, the indicator may be a set ofvalues each indicating which parameter from a set of parameters of eachtype to use or may alternatively be a single indicator which refers tomultiple parameters of different types. For example, the functions maybe quantization parameters and multiple different types of statisticsused for decoding each data structure, where each data structurecomprises different types of symbol encoded using different encodingentropy encoding techniques. This will become clear from the examplesbelow.

Table 1 shows conceptually how an indicator may refer to multipleparameters. If the bytestream signals that an indicator should be usedfor a particular subset then the decoder may use that indicator or indexto identify which of a set of parameters should be used for that subset.

TABLE 1 Statistics Metadata Quantization Statistics Metadata Indicator#1 Parameters #2 a 1 1 1 b 1 2 2 c 1 1 2 d 2 3 3 e 3 3 3

As shown in Table 1 for example, an indicator value of a may correspondto a first set of statistics 1 for one type of symbol from a set ofthree possible statistics metadata (1,2,3), a set of quantisationparameters 1 from a set of three possible quantisation parameters(1,2,3) and a set of statistics 1 from a set of three possiblestatistics metadata (1,2,3). Thus the decoder is able to identify whichof a set of three should be used for each parameter when decoding aparticular subset of the bytestream. It will of course be understoodhere that the numbers are merely exemplary.

FIG. 1 illustrates conceptually the principle of the attributeindicator. On the left of FIG. 1 are shown five subsets of a bytestream.Each of the five subsets corresponds to a data structure. Within abytestream, there may be included an indicator value. Here again we showfive index values. Subsets A and B are each associated with indicatorvalue a. That is, indicator value a is sent and associated with subset Aand B. Similarly, subset C is associated with indicator value b, subsetE is associated with indicator value c and subset D is associated withindicator value e.

Each of the indicator values point to a set of parameters which can beused for decoding. For example, indicator a points to parameters{1,1,1}. Indicator b points to parameters {1, 2, 2}. Thus, within thebytestream the indicator is signalled and the decoder can interpret thisindicator and retrieve a set of parameters according to that indicatorfor use in decoding each subset. Thus, within the bytestream, thedecoded will identify the indicator a that corresponds to subset A. Thedecoder will retrieve the parameters {1,1,1} in order to appropriatelydecode the subset A from the bytestream.

Each indicator in the bytestream may be difference encoded. That is, theindicator value may be calculated based on the difference between thecurrent indicator and the previous indicator. For example, if a firstdata structure A has an indicator a, then the indicator valuetransmitted in the bytestream for data structure B may be sent asindicator value b-a. Where data structures correspond to neighbouringsections of the information to be compressed, the statistics are likelyto be similar. Therefore, the difference encoded indicator value may infact be 0 where the statistics are the same, which reduces the metadataneeded to be transmitted to signal multiple decoding parameters to thedecoder. This significantly reduces the size of the bytestream. Anexample of where this is useful may be for example an image of the skywhere neighbouring data structures each have a similar profile of bluedata and are likely to have similar decoding parameters. To decode theseindicator values, the decoder must store the previous indicator valueand sum the indicator values to derive the indicator value associatedwith the subset in question.

There are numerous advantages of the described technique. For example,the technique provides for the reduction in metadata signalling—itbecomes possible to signal decoding parameters only once while stillenabling differential encoding of subsets of the bytestream.Additionally, it becomes possible to group together spatially differentbut statistically similar clusters or regions. This is an improvement onany statistical-based clustering technique which possibly providesstatistics based on a grouping of regions within a grid. Further, bygrouping the statistics together, entropy encoding efficiencies can beachieved.

By selectively sending quantization parameters grouped by an indicator,different regions of information can be selectively quantized. Thisenables regions which are important to be quantized to a higher quality,for example where there is an edge in the information or where thereaders eye might be drawn to a low quality, importantly, withoutintroducing an overhead to the data size as the quantization parameterscan be sent only once and referred to by the indicator for a particularsubset or region.

Where the subsets are grouped together, multiple different types ofparameters can be used and signalled for each region using only onereference (i.e. index). This further reduces the metadata overhead.Similar regions are likely to have similar statistics and quantizationparameters, which further reduces the overall bytestream size.

Another advantage lies in the ability to, without increasing themetadata overhead, make frame, plane or level of quality based decisionson the encoding and decoding. Finally, selective entropy encoding ispossible to improve decoding speed and or detail.

The concept of an attribute indicator which informs the decoder of a setof attributes or parameters used to decoding subsets of a bytestreamwill now be described in the context of a technique for hierarchicallyencoding a sparse plane of an image or video into a data stream andrecreating that plane from an encoded data stream so that the datastream can be separated into subsets with each subset being decodedseparately.

A hierarchical structure is proposed comprising tiers. For a tier of afirst type, a first type of data element is defined comprising a subsetof the set of encoded data; and for a tier of a second type, a secondtype of data element is defined comprising one or more attributesassociated with the first type of data element. The location of thefirst type of data element in the recreated array is determined from theinformation contained in the second type of data element.

By defining the data structure required to map the instances of whereresidual data occurs within a frame of data, it is possible to provide afaster decoding and encoding methodology. Furthermore, the data, andmetadata describing the structure of the data allows for individualportions of the frame to be selectively decoded without reference toother portions of the frame. Thus it is possible to parallelise thedecoding process. The technique further reduces the data in memory, forexample by allowing data in memory at an instant to be a subset of thedata for the whole plane.

Typically the data can be of any nature as long as the values can bemapped into an array, although the techniques are also applicable tolinear data and most beneficial for image reconstruction. In the case ofa picture or video, the data could be values associated with a colourspace (e.g., the value of a red component in an RGB colour space, or thevalue of a Y component in a YUV colour space, etc.), or alternativelythe data could be residual data (whether transformed or not) or metadataused to decode a bytestream. Residuals are further defined in thepresent application, but in general residuals refer to a differencebetween a value of a reference array and an actual array of data.

It should be noted that techniques described in the followingdescription are agnostic as to the meaning or use of the decoded array.Rather the concept of decoding a sparse array from an encoded bytestreamis discussed, for example. Of course, the data set may be used toreconstruct a larger dataset by combining multiple decoded data sets.Once recreated the data may represent any information which has beencompressed, such as an image or sonogram.

As implied above, the techniques described here relate to the generalprinciple of separately decoding data structures using informationcontained within a different data structure. An alternative way ofconsidering the techniques described is that the techniques provide forthe “breaking-up” or deconstruction of a graph to allow for separatedecoding of sections of the graph. That is, an unbroken graph can bebroken into a series of graphs or separate data structures which can beseparately decoded.

The techniques described herein provide for the benefits that when suchbroken data structures are sent separately they can be correctly putback together in the graph and further that the total data transmittedcan be reduced by implicitly signalling data structures which representconsistent values, e.g. sparse data.

To provide these benefits the technique maps subsets of a datastreamonto a data structure, such as a tree. In other words the techniquecreates a data structure of ‘pieces’, the data structure storing thespatial information. If the data structure is a tree, the techniquecould be considered to be a hierarchical ‘tree of pieces’. To recreatethe data, the pieces can be mapped to a data structure to create a datastructure of the pieces which stores the spatial information. Theprocess can be recursed for more tiers in the hierarchy.

FIG. 2A illustrates an exemplary representative bytestream or bitstream.These latter terms will be generally used interchangeably throughout thespecification. The illustrated bytestream is simplified andrepresentative of an implementation only. The bytestream is divided intoa root tier and a Top tier. The root tier contains attribute metadatarelating to the Top tier. The Top tier contains data elements which mapto the image to be recreated. An intermediate tier, not shown, may beprovided where the root tier contains attribute metadata relating to theintermediate tier and the intermediate tier contains attribute metadataof the Top tier. As will be understood from the description below theorder of the elements is representative only for understanding thetechnique.

FIG. 2B represents the data structures of the Top tier which will hereinbe referred to as the top tier, data tier, tile tier or tier 0,interchangeably. The bytestream of the data tier comprises a series ofdata structures, each comprising structure metadata and data elements.The data elements represent the information that has been compressed. Asillustrated in FIG. 2B, certain parts of the information may not beincluded in the bytestream and may be implicitly signalled. The datastructures of the data tier each correspond to a block of the array, ortile, as illustrated in FIG. 2C. FIG. 2C shows how each of the blocksmap to a region of the array and that certain regions may not beincluded in the bytestream and instead are implicitly signalled.

The proposed tiered structure is visualised in FIG. 2 in a bottom-to-topstructure. It will be understood that this is merely a visualisationused to aid in understanding the principles of the invention.

The hierarchical structure defines instances of when data is present(and therefore needs to be encoded) and provides a mapping to identifywhere such data is present.

The root tier contains a set of attribute metadata elements whichindicate attributes of the subsequent tier. Based on the attributes, thedecoder can identify if a data structure is included in the subsequenttier. Each attribute metadata element of the root corresponds to, ordescribes, a location of a data structure of the subsequent tier, thatis, the attribute metadata element may include information about a datastructure in the subsequent tier or may indicate that no data structureis included in that predetermined location. A defined mapping of theelements maps to a location in the data structure.

The root tier may optionally be a data structure itself. That is, thedata structure may comprise structure metadata which describes theattribute metadata elements. In other words the data elements of thedata structure are metadata (or attribute metadata elements) whichrelate to the subsequent tiers and the data structure comprises metadata(or structure metadata) which describes the metadata (or attributemetadata elements). The root tier may be referred to as a metadata tier.A metadata tier preferably contains metadata but no data, whereas a datatier contains metadata and data.

The root and intermediate tiers each demonstrate similar functionality.That is, the attribute metadata elements of the root tier each describea data structure of the first tier (or lack thereof) and attributemetadata elements of the first tier each correspond to a data structureof the second tier (or a lack thereof). In other words, each attributemetadata element corresponds to a sub-grid of the overall grid to bedecoded. That sub-grid either being represented by an additional tier ofmetadata elements, a plurality of data structures having data elementsor that sub-grid may be void of data. As will become clear, void hererefers to an area of the grid having a consistent value.

In the exemplary data structure and technique described, a plane of datais divided into a series of tiles, each having a set of dimensions.Throughout the present description we use the size 16×16 but this may beany size N×N. Depending on the size of the plane to be decoded, only aroot metadata tier may be needed. For example, a root data structurestoring attributes in a 16×16 array may correspond to 256 datastructures in the top tier. If the top, or data, tier contains 256 datastructures, each storing a 16×16 array of data symbols, then a 256×256array can be mapped. The addition of an intermediate tier provides forthe possibility of mapping a 4096×4096 array of data. If the array is aplane of an image, planes of data are suitable to decode UHDTV videostreams.

The term tiles may be used interchangeably throughout the description torefer to the data structure of the data tier which is a subset of thebytestream used to represent the data values of the information to becompressed. The tile corresponds to a region of that information (or agrid) and comprises data symbols that may or may not be quantized andtransformed and associated with corresponding metadata elements withinthe same data structure.

As indicated, each data structure of the data tier corresponds to asection of the array. It is contemplated that each data structure of thedata tier may be decoded separately and in parallel. However, the datatier may possibly not be located within the array until the previoustiers have been decoded. Thus the technique provides a combination ofserial and parallel processing.

Once the data structures in the data tier are decoded, the data elementsof the data structures are optionally each mapped to the array in apredetermined order. A predetermined value is inserted in theinformation where the attribute metadata elements of the tiers belowindicated that there is a void area. That is, when the data structuresare arranged as visualised in FIG. 2 the attribute metadata elements ofthe previous tiers indicate there are no data structures sent in thebytestream for that position of the array. Each attribute of the lowertier describes a data structure (or lack thereof) in a particularlocation in the immediately higher tier.

The attributes stored in the data structure of the metadata tiers, i.e.the root tier and tiers-k, may comprise one or more of the following,non-exhaustive, list of attribute metadata elements:

-   -   a positive or negative flag indicating if a corresponding data        structure exists in a subsequent tier;    -   the dimensions of a corresponding data structure in a subsequent        tier;    -   information to enable the decoder to locate a corresponding data        structure in a subsequent tier in the bytestream, such as:        lengths of streams; an offset from the current location in the        stream; or, a fixed location in the stream;    -   information to facilitate entropy decoding of a corresponding        data structure in the subsequent tier such as indication of        parameters to use in a range decoding operation; and,    -   other parameters or attributes associated with a corresponding        data structure in a subsequent tier in the bytestream such as        quantization parameters or a predetermined value to be used        where no data symbols are included in the data structure.

According to the present invention, instead of explicitly signallingeach attribute metadata element, the attribute metadata element may bean indicator, or index, which refers to a set of attributes orparameters to be used. For example, a single indicator may be used tosignal all of the dimensions, the statistics and the quantizationparameters for the data structure of the data tier. The indicator andits corresponding attributes may be sent separately in the bytestream orbe predetermined by the decoder by retrieving the attributes from astore. Thus, a data stream need only to send the indicator and itscorresponding parameters (or attributes) once and then subsequentlyre-use those parameters throughout the bytestream, thus reducing themetadata overhead.

For completeness, the parameters themselves may be signalled in thebytestream separately and referred to by an indicator value so that theyare only signalled in the bytestream once or they may be pre-coded intothe decoder or may be retrieved from a separate store by the decoder andpointed to by the indicators signalled in the bytestream.

As mentioned above, each subset of the bytestream may be a datastructure. It has previously been described in this document howmultiple data structures may be sent within a bytestream andhierarchically encoded in order to separately decode each of theplurality of data structures. Thus, according to an example orimplementation, the indicator value associated with each subset may betransmitted within the attributes metadata element of the lower tier.Such a concept is illustrated conceptually in FIG. 3.

FIG. 3 illustrates a root tier of a pyramid of pyramids in which the toplayer of the first pyramid points to a data structure of a higher tier.Using the same notation of FIG. 2, the values in the top layer of theroot tier are the indicator values here [a,a,b,e,c]. Once decoded, theseindicator or index values can be used to retrieve a set of decodingparameters to decode each of the subsets of the higher tier. The conceptillustrated in FIG. 3 is that once the indicator is retrieved, each ofthe decoding parameters is retrieved from a database store in order toretrieve the parameters suitable for decoding that data structureindicated by the attribute indicator and the location of the attributeindicator within the hierarchical data structure.

The invention herein may be applicable to decoding any type of datastructure. For example, any structure of decoded symbols. The indicatormay be used to point to parameters to facilitate decoding of thosesymbols.

It is contemplated that each data structure may be any suitable datastructure for encapsulating the necessary information and each of theseattribute elements may be stored in the data structure in any knownmanner such as a tuple. For example, the attribute metadata in the roottier may be a relatively simple set of encoded bits, each indicating theattributes of the next tier. Similarly each data structure of the datatier may optionally be a tree to store the necessary data elements orany suitable data structure combination of metadata and data symbolsused to encode the information.

The data structure of the top tier may preferably be a sparse quadtreeas defined in GB1812407.3, the content of which is incorporated byreference in its entirety. The term sparse quadtree will be usedthroughout the present invention to refer to a data structure asspecified in this document. The structure will be summarised below. Thedata elements of each N×N tile may be accurately, spatially locatedwithin the array when the tile is mapped to the array and the dataelements which correspond to the sparse information within each tile maynot be included, i.e. signalled implicitly, within the data structure.

Similarly, the data structure of the other tiers may also be a sparsequadtree. In this way, the attribute metadata elements may be the dataelements of the top layer of the sparse quadtree which are eachassociated with (or describe) a data structure of the top tier. Wherethe sparse quadtree of the root or metadata tiers indicates that noattributes exist in the data structure in the top layer for a particularsection of the data structure, optionally, the technique may insert apredetermined attribute that indicates that there is no data structureincluded for a corresponding part of the data tier. Alternatively, thatattribute may be assumed or is implicit, thus signalling that nocorresponding data structure is included in the top tier for that partof the array.

Alternatively, each quadtree of the root and first tiers may be a densequadtree with the quadtree structure being used to locate the attributesin the top layer spatially to generate the higher tier but may notsignal information implicitly.

In an implementation, each data structure may thus be an abstract datatype, optionally of the sparse quadtree data type.

It will thus be understood that the benefits of the concepts describedherein may be realised with any data structure in each tier.Nevertheless for clarity, the preferred implementation of the conceptsin which the sparse quadtree is used in all tiers will be describedthroughout the examples given in the present application.

In the sparse quadtree data structure, once the tree is built, the TopLayer of the tree, or the final layer, may include data symbols. Theorder in which the data symbols are included in the tree represents thespatial information of an array. The sparse quadtree may retrieve anydata element in the Top Layer depending on the tier in which it is used.For example, the sparse quadtree data structure may include a list,tuple, set or integer.

When the sparse quadtree data structure is used for the data tier, thebytestream will include data symbols to recreate an array and in themetadata tiers may include a tuple representing the attributes. As thatarray is located within the tiered structure defined above, the array islocated in a larger array from the tiered structure.

We are illustrating the concepts using a quadtree to recreate a 16×16array of data and therefore there are four layers and a root in the treegiving 256 possible leaves, each representing a value in the 16×16 gild.Other sized grids may utilise different ordered trees.

The following sets out an example process of decoding an exemplarysparse quadtree data structure. During the process of decoding, anordered tree is built. Code symbols from the bytestream are converted todecoded symbols and attached to nodes of the tree. The data structureintroduces a special symbol which is used by the decoder to build thetree. We refer to this special symbol here as a node symbol. The nodesymbol indicates to the decoder how to build the tree. Within the nodesymbol is information which tells the decoder how to map the informationfrom the bytestream to the tree and what it can expect in thebytestream. Using a specific traversal order, the decoder maps the nodesymbols to the tree and can subsequently map the data received in thebytestream to leaves of the tree in the correct locations. The spatialinformation or the order of the original information is then containedwithin the tree. The mapping of the node symbols and traversal leavesblank spaces in the tree which can be simulated or inferred to indicatethat a predetermined value was in that location in the originalinformation but was not sent in the bytestream.

The node symbol is a series of binary values or flags that indicate tothe decoder if a particular branch in the tree has an expected childwhere the branch has an expected child if there is a node included inthe data set for the Top Layer descendants of that branch. That is, thebytestream contains information on the existence of a child node or not.\Mien the decoder traverses the tree to reach a leaf (a node in the TopLayer), the bytestream contains a series of elements, such as four datasymbols or four attribute metadata elements (e.g. a tuple), eachrepresenting a value of the leaf of the tree. The tree can besubsequently mapped to a grid using a defined order with each leaf onthe tree corresponding to a location in the grid. If the sparse quadtreeis a structure in the metadata tier, the attribute metadata element isnot mapped to a location in the grid but instead is mapped to the datastructure.

Within the bytestream, the node symbols of the data structure areinterspersed. That is, the node symbols and data symbols (or attributeelements) occur between or amongst one another within the bytestream. Afeature of the data structure is that the decoder cannot know the orderof node symbols and data symbols (or attribute elements) prior to thedecoding process. Thus there is no set or predetermined ordering to theinterspersal of the information. The location of the data symbols (orattribute elements) is deduced from the information contained within thenode symbols. The node symbols and data symbols (or attribute elements)may not occur within the bytestream one by one or regularly but ratherwill be present within the bytestream irregularly and sequentially, butnot randomly.

The same traversal is used in the decoding as the encoding to ensurethat spatial information is retained. Thus, the sparse quadtree datastructure defines the instances and location of the elements. Preferablythe tree is traversed according to a depth-first pre-order traversaland, in the data tier, the data symbols are mapped to an array accordingto a Morton order but any other order may be used provided the encoderand decoder agree. In the metadata tier, the Morton order can be used toassociate the attribute metadata elements with the data structures ofthe subsequent tiers.

FIG. 4 illustrates an image that will be used throughout the presentdescription. The image reflects both the data to be encoded as well asthe reconstructed image. It will of course be understood that the imageis a simplified plane of data that could represent residual data orcomponent data of an image.

As can be seen, there are four regions of the image that are void ofdata, that is, are entirely sparse, and eight regions that contain dataelements.

In the exemplary data structure and technique described, the plane ofdata is divided into a series of tiles, each having a set of dimensions.Throughout the present description we use the size 16×16 but this may beany size N×N. In this simplified example, only a root tier and top tier(or data tier) may be included in the bytestream and accordingly we willomit discussion of any intermediate metadata tier.

FIG. 5 shows schematically the 12 grids that may hold data of theSurface of FIG. 4, where the term Surface represents a plane of data orother large array of data. FIG. 5 also depicts the bytestream.

First, the start location of the root tier may be signalled in a headerof the stream. This may take the form of an offset A from the start ofthe bytestream as shown in FIG. 5.

Second, the decoder may start to read a data structure from the streamcorresponding to the data structure of the root tier.

The decoder may start to read a data structure with a dimensions tuple(4,3) which contains metadata about all the 12 gilds. The dimensions maybe signalled separately, either in the header or elsewhere. The conceptof dimensions of the grid will be discussed in more detail below. Aswill be clear from the discussion below, the signalled dimensions may beof the root data structure in which the root data structure thencontains the dimensions of the subsequent data structures or may bederived from a signalled dimensions tuple of the plane of data that isencoded in the hierarchical structure.

FIG. 6 illustrates the data structure of this example and showsschematically the structure of the part of the bytestream starting atoffset A and describing the root tier. It will be seen that thebytestream comprises three node symbols followed by metadata about datastructures that possibly convey actual data. In this diagram, YES standsfor metadata that indicates use of the bytestream to convey data and NOstands for a void data structure, absent from the bytestream, since itsentire data are implicit zeros.

Some digits of the node symbols in FIG. 6 have been masked with an “x”.This is connected with the dimensions of the array (4,3) being smallerthan (16,16) which is the dimensions of the quadtree made possible bythe sparse quadtree data structure having 4 layers. Masked digitsindicate that the corresponding child itself is inactive andautomatically has zero children. The values of these masked digits playno role in decoding and can be optimized by an encoder to minimizeentropy.

FIG. 6 is supplemented to show how the attribute metadata elements mayalso comprise an index value which refers to the decoding parameterssuitable for decoding a particular subset.

In the following description active will mean unmasked and representingdata in the compressed information and inactive will mean masked andincluding optionally transmitted data outside the information to becompressed.

FIG. 7 illustrates the process by which the data structure of the roottier is built from the bytestream. The process begins to build a treeand assigns a root node. The node symbol [boa] is read from thebytestream. The first flag, 1, indicates that further information forthis branch is included in the bytestream. The three masked flags are aresult of the other branches of the root node of the data structurebeing inactive.

The decoder detects inactivity from the signalled dimensions andimplements masking by a rule during depth-first discovery. The decodermasks bits in the first node symbol that correspond to inactivechildren. For an active child, it assigns the appropriate quadrant,every time a new active node is placed.

Following a depth-first pre-order traversal (as pre-agreed with theencoder) the traversal visits to the next node of the tree. Againreferring to FIG. 6, we see that the next node symbol is [1xxx]. As withthe root node, this gives one active branch and three inactive branches.

The depth-first pre-order traversal then visits the next node. The nextnode symbol is read from the bytestream which is [1111]. This indicatesthat there are four branches in this layer of the tree, each having achild and grandchild node.

In the data structure of a sparse quadtree, the next layer of the treeis optional. The node symbols that correspond to the penultimate layerin the tree are not sent but rather are implicitly assumed by thedecoder. It will be understood that the node symbols in this layer wouldbe understood to include a set of positive flags (or [1111]). Anotherway of looking at this implicit signalling feature is that a node symbolis sent or included within the bytestream only if a grandchild nodeexists for the visited node of the tree. In other words, a node symbolshall have a grandchild node.

Accordingly, the decoder knows that the next symbol to be expected fromthe bytestream is an attribute metadata element representing anattribute of a data structure of the subsequent tier, as it has reachedthe top layer of the tree following the depth-first pre-order traversal.

Since we are in the root tier, the decoder retrieves the attributemetadata element (for example a tuple of attributes) from the bytestreamand associates the element with the node in the top layer. Following thedepth-first pre-order traversal and the implicitly signalled penultimatelayer, the next three elements to be decoded from the bytestream willalso be attribute(s) of the data structures in the subsequent tier.

Descending the tree the decoder will see that the node symbol was [1111]indicating that the next branch is active and there are data symbols inthe bytestream for this branch. The decoder ascends the tree until theTop Layer is reached. Here again, the next four attributes will beretrieved from the bytestream and associated with the next four nodes ofthe tree.

Descending the tree the decoder will see that the node symbol was [1111]indicating that there are data symbols in the bytestream for thisbranch. However, the signalled dimensions imply that only two of thebranches of the penultimate layer are active. Accordingly, the decoderwill retrieve only two attributes from the bytestream.

Again, descending the tree the decoder will see that the node symbol was[1111] indicating that the next branch is active. However, the signalleddimensions imply that only two of the branches of the penultimate layerare active. Accordingly, the decoder will retrieve only two attributesfrom the bytestream. This section of the top layer has now been filled.

Following the depth-first pre-order traversal, the decoder descends toLayer-3. Since this layer only indicates one branch and this branch hasalready been visited, the process of building the ordered tree will endas all nodes have been visited and populated.

FIG. 7 shows 10 inactive nodes (on dashed branches). During discovery ofthe tree's topology, metadata as well as node symbols were encountered.This information (or attributes) about other data structures is passedthrough as elements which are depicted (unchanged from FIG. 5) abovecertain top layer nodes of the tree.

These elements are now allocated to a 3×4 pattern as shown in FIG. 8.The scan order is called Morton order and is illustrated in FIG. 5. Notethat four vertices on the path shown in FIG. 5 are inactive. This isreflected in FIG. 7 by the corresponding inactive nodes in the Top Layerof the tree.

The values in the stream shall be interleaved in the example indepth-first order. In the example, the data of the tree is mapped in aMorton ordering. A Morton ordering maps multidimensional data to onedimension while preserving locality of the data points. It wasintroduced in 1966 by G. M. Morton. The terms Z-order, Lebesgue curve,Morton order or Morton code are used in the art.

Morton ordering is well known in the art and will be understood. It willalso be understood that any suitable mapping of the data from the treeinto the grid may be utilised.

In practice Morton ordering using 2×2 blocks means that the symbols ofthe tree are mapped to the grid in the following example order for an8×8 grid:

0 1 4 5 16 17 20 21 2 3 6 7 18 19 22 23 8 9 12 13 24 25 28 29 10 11 1415 26 27 30 31 32 33 36 37 48 49 52 53 34 35 38 39 50 51 54 55 40 41 4445 56 57 60 61 42 43 46 47 58 59 62 63

When considering the mapping of the tree to the pattern, it can beconsidered that the z-order mapping results in each branch of the treebeing a quadrant of the overall array.

While a Morton ordering is a preferred ordering, it is also contemplatedthat other orders such as a Hilbert space-filling curve, also known as aHilbert pattern or Hilbert curve, may be used which may provideimplementation or efficiency gains depending on the array to becompressed and the likely locations of non-zero elements in the array.In certain circumstances the Hilbert curve ordering will also havebetter locality preserving behaviour.

FIG. 8 illustrates how the pattern of attributes mapped from the roottier corresponds to data structures received in the bytestream and theorder in which the data structures are received in the bytestream.

At this stage of the process, the decoder knows the order of each of thedata structures of the top tier in the bytestream. The decoder knowswhere there are void areas of the array where no data structures areincluded in the bytestream and accordingly knows how to arrange the dataelements of the data structures once the data structures have beendecoded.

Accordingly, each of the data structures in the data tier can now bedecoded and their encoded data added to the array.

In decoding this data structure, the decoder follows the sparse quadtreedecoding technique. The reader will be reminded that while it isdescribed that the data structures of this tier are sparse quadtrees,the data structures may be any suitable data structure suitable forspatially locating data elements within in an array. From the decodingprocess above, we know where the data of this data structure will bestored in the wider array. The sparse quadtree data structure of thisexample stores a 16×16 array of data and accordingly includes fivelayers: a data or top layer, layer-1, layer-2, layer-3 and a root layer.

As illustrated in FIG. 6, the attribute metadata for the fourth elementin the bytestream is an index value {123}. When this is mapped into thetree of FIG. 7, it can be seen that the first node in the outer layerwill have this corresponding attribute metadata element. Using thepredetermined mapping order, and as illustrated in FIG. 8, theattributes can be mapped to a corresponding data structure retrieved inthe bytestream and that bytestream decoding using the indicatedparameters.

Returning to FIG. 4, this figure shows the reconstruction of all theTiles into a decoded image of a horse. It will be recalled from theMorton Order of illustrated in the path of FIG. 4 that this is thebottom right and is included last in the bytestream, starting fromposition Y of the bytestream shown in FIG. 8. For brevity we have notdescribed in detail the content of the bytestream between the end of theroot tier at position R to the beginning of the final data structure atposition Y. The content of these can be deduced from the horse image andfrom following the process describes above.

In order to decode each data structure in parallel, the start locationof each data structure to be decoded may need to be known. In certainembodiments, the data structures of the top tier may be of a fixedlength so that the decoder may predict the locations of each structure.Alternatively, the start locations of each structure may be separatelysignalled. In preferred embodiments the attributes of each data elementincluded in the lower tiers indicate to the decoder how to decode thecorresponding data structure. For example, the attribute may include thestart location of the data structure within the stream and also mayindicate that the data structure is not included at all as was explainedabove. The attribute may accordingly be a 0 value if the correspondingdata structure is not included in the stream and may be a positive valueindicating the start of the data structure.

It has been described above how optimally a data structure may beconstructed and decoded to incorporate a set of interspersed nodesymbols and data symbols. Once the symbols have been output they may besubsequently entropy encoded. The encoded stream may be entropy decodedbefore the set of symbols are processed. For example, the symbols may bedivided into codes which are then encoded using a Huffman encoding anddecoding operation. Alternatively, the stream of symbols may be encodedand decided using an arithmetic coding operation, such as a rangeencoding and decoding operation. These and other similar entropy codingtechniques are well known in the art.

Entropy coding is a type of lossless coding to compress digital data byrepresenting frequently occurring patterns with few bits and rarelyoccurring patterns with many bits. In broad terms, entropy codingtechniques take an input codeword and output a variable-length codewordusing the probability of the input codeword occurring in the data set.Therefore, the most common symbols use the shortest codes. Theprobability information is typically stored in metadata used by thedecoder to recreate the input information from the output codeword.

The attributes may include probability information directly or mayalternatively, and preferably, indicate to the decoder which of a set ofprobability information should be used to decode that particular datastructure referenced by the attribute. The probability information maybe stored in the decoder or may be signalled separately in thebytestream.

An entropy decoding operation may retrieve metadata from a store ofmetadata corresponding to the attribute signalled. The metadata mayinclude decoding parameters for example and may include an indication ofprobability. For example if the decoding operation is a range decoder,the metadata may include a probability distribution or cumulativedistribution function.

In an implementation, one entropy decoding operation may sequentiallydecode the data structures or multiple entropy decoding operations ormodules may be used. In practice, each data structure may be decodedusing a different type of entropy decoding for example, the attributesmay indicate that a data structure is encoded using Huffman encoding anda separate data structure is encoded using arithmetic encoding.Preferably though, a range decoder is used with the attributesindicating to the range decoding module used to decode the datastructure, which set of metadata should be used to decode that datastructure.

The following describes an improved and innovative technique for entropycoding a bytestream. Immediately above we described how the process ofdecoding, once performed can then be applied to a process ofde-sparsification to identify sparse areas of an array and accuratelylocate values in the array. The described operation couples thede-sparsification and decoding steps together.

A high level overview 900 is shown in FIG. 9. After the bytestream isdecoded in a decoding operation 901, an output plane of data undergoes adequantization 902 stage and a composition transform 903 stage. Thecomposition transform stage 903 and de-quantisation stage 902 are knownin the art. For example the composition transform stage 903 may includea directional transform of a plane as described in WO2013/171173 or awavelet or discrete cosine transform. In these examples, thequantisation parameter may be signalled for that data structure by theattribute metadata elements which include an index value.

It is described herein that the decoding operation 901 may include twostages, that is, an entropy decoding stage 904 and a de-sparsificationstage 905. The stages of the decoder are coupled together and areinterrelated so as to efficiently identify the compressed information.The entropy decoding stage acts to decode a symbol from a stream ofdata. The de-sparsification stage acts to analyse the symbol and informthe entropy decoder what type of symbol is next to be decoded.

In preferred embodiments, the de-sparsification stage or module 1705builds a tree as described above. The de-sparsification stage receives asymbol from the entropy decoder and builds the tree. Thede-sparsification stage then, from the process of building the treeinforms the entropy decoder what type of symbol to expect next, i.e. anode symbol or a data symbol. By analysing the node symbols in themanner described, the de-sparsification stage can identify that the nextsymbol will be a node symbol or a data symbol by following the treetraversal and identifying that no data symbol is expected for a branchof the tree where the node symbol includes a flag indicating as such.

The terms de-sparsification stage, de-sparification module andde-sparsifier may be used interchangeable throughout the presentdescription to refer to the functionality of the module. Similarly, theterms entropy decoding stage, entropy decoding module and entropydecoder may be used interchangeably to refer to the functionality ofthat module. It will of course be understood that the functionality maybe provided by a combined module or multiple sub-modules.

At the entropy decoding stage, the module has access to multiple sets ofmetadata used to decode different types of symbols using the entropydecoding operation.

The attribute index value indicates to the entropy encoding stage whichmetadata should be used for the different types of symbols. For example,the index may point to a first set for a node symbol and a second setfor a data symbol.

First, the entropy decoding stage will first decode a symbol using afirst set of metadata. The entropy decoding stage will then send thatsymbol to the de-sparsification stage. The entropy decoding stage willthen wait to receive an indication of the type of symbol that is to beexpected next. Based on the received indication, the entropy decodingstage will use a respective set of metadata according to the type ofsymbol expected in order to decode the next symbol using entropydecoding. In this way, different metadata can be used to decode a dataset even when the data within the data set does not follow apredetermined pattern and the different symbol types are irregularlyinterspersed within the original data to be encoded or reconstructed.

Thus, here the index value will refer to two sets of metadata fordecoding that data structure. The index will be set at the encoderaccording to which two sets are best for this data structure. Using theterminology above, the data structures are grouped according to theirsimilar characterising and index sets for each group. Thus, only anindex needs to be set. The metadata needs only to be set once or storedand not set multiple times.

It will of course be understood that instead of using one entropyencoder and multiple sets of metadata the system may instead utilisemultiple entropy encoder modules for each type of symbol to be decoded.For example, the desparsification module may instruct a different moduleto perform an entropy decoding operation based on the type of symbol itexpects next in the dataset. Thus, the index could indicate theparameters to be used for each decoder or which decoder is to be usedfor each symbol. This also applies where there is only one type ofsymbol in a data structure.

The process will now be described. We start by assuming that the firstsymbol in the stream is of a first type. In the implementation it is notrelevant if the de-sparsification stage instructs the entropy decodingstage that the first symbol is of a first type of the entropy decoderinherently has a degree of intelligence or predetermination to identifythe first expected type.

The entropy decoding stage will retrieve metadata from a store ofmetadata corresponding to the first symbol type: the metadata beingchosen according to the index value. The metadata may include decodingparameters for example and may include an indication of probability. Forexample if the decoding operation is a range decoder, the metadata mayinclude a probability distribution or cumulative distribution function.The index will thus refer to which cdf to use for that symbol. Theoperation of a range decoder in the context of the present disclosurewill be described below in the context of FIG. 10.

As indicated above, there may be a degree of intelligence orpredetermined expectation coded into the entropy decoding stage ormodule. For example, when it knows a data symbol is to be retrieved itmay know to retrieve four. Depending on implementation, the entropydecoding stage may of course wait for an indication of the symbol typeeach time it tried to identify a symbol from the stream.

It is recalled that rather than the entropy decoding stage switchingbetween sets of metadata, there may instead be multiple entropy decodingmodules, each using one set of metadata and each retrieving a symbolfrom the stream of interspersed symbols of different types according towhich type of symbol is to be expected next.

It was described above that the entropy decoding stage may be any typeof entropy decoding module. For example, the entropy decoding module maybe a Huffman decoding module where the symbols in the stream are of afixed length. Preferably however the entropy decoder is a range decoder,the operation of which will now be described in the context of FIG. 10.If multiple decoding modules are used, the first type of symbols may bedecoded using a first type of entropy decoder and the second type ofsymbols may be decoded using a second type. For example, the fixedlength nodes symbols may be decoded using a Huffman decoder and the datasymbols may be decoded using an arithmetic decoder, which may bebeneficial if the types of symbol are of differing lengths or is onetype lends itself to a fixed length operation and the other to avariable length operation.

FIG. 10 illustrates a range decoding module for implementation in thepresent system which may perform incremental decoding. FIG. 10illustrates a sequence 200 of bits. The range decoder 1000 takes aportion of this sequence and updates an internal state register 1001.

To begin the decoding process, the decoder must have enough informationto decode the first symbol unambiguously. The decoder controller willretrieve a first set of metadata from a store of metadata 1004, themetadata including a first cumulative distribution function whichmetadata to use is indicated by the index value. From the metadata, therange decoder can identify the smaller number of bits it needs from thestream to identify the node symbol with unambiguous probability, shownwith a dotted line here in the sequence 200.

The range decoder sequentially analyses increasingly larger portions ofthe stream until it can be confident of the probability of the symbolthat has been encoded. That is, the decoder compares the increasinglylarger portions of data to the cumulative distribution function bynarrowing the possible symbol with every portion of data analysed. Oncethe symbol has unambiguously been identified, the symbol is output 1905.

The controller 1002 of the range decoder will now wait until it receivesa trigger 1002 to decode the next symbol. The trigger 1903 may includethe type of symbol to be decoded and the decoder will choose for the setor may indicate the metadata to use based on the index value. From thisinformation, the decoder will retrieve the respective metadata from themetadata store 1004 which will include a cumulative distributionfunction for the symbol to be retrieved next.

From the cumulative distribution function, the decoder will identify ifit should update its state (or tag) and either read in more bits fromthe sequence 200 or shift out bits 1006 (i.e. or otherwise discard bitsinto trash). For example, the decoder will identify if the mostsignificant bit of the state (or tag value) is the same or differentbetween the upper and lower limits of the possible values and whetherthe amount of bits it holds in its state is sufficient to identify thesmallest or most infrequent value in the distribution function. If themost significant bit is the same between the upper and lower limits ofthe potential symbols, that most significant bit can be discarded.

Schematically, the image of FIG. 10 shows how the state may show thatthe data in the stream may be retrieved from a later part of the encodeddata stream and that the next identified symbol may not be the initialbits of the data stream but may be retrieved from a latter part of thedata stream. Such eventuality is well known to the skilled addressee whowill be familiar with range encoding techniques. FIG. 19 merely attemptsto demonstrate that the range decoding technique incrementally reads asequence of bits and from that sequence identifies a symbol based on aprobability function, preferably a cumulative distribution function andthat the functions may be pointed to by an index value so as to groupdata structures together. The symbols may not be included within thesequence of bits in a strictly sequential manner.

While we have not described the functionality of a range decoder forbrevity, we believe it is well known in the art. However, thefunctionality of the system is clear. The range decoder of the presentdisclosure is configured to use a respective cumulative distributionfunction according to a type of symbol expected next in the bytestreamand form an index value indicated for that data structure.

For example, given that the final symbol can be recognized from thepreviously decoded interspersed symbols, a range decoder may be able towork with just the location of the start of the data structure. It ispossible to define the start in terms of an offset from the start of theprevious stream. Nevertheless, there are certain implementations ofrange decoder where reading data from the bytestream stops before thefinal symbol or symbols are deduced (not deduced at all, or not entirelydeduced) and a final series of calculations is performed. In such a caseit may be necessary to know where the structure ends, so that thehandover to the final calculations can be done at the right stage indecoding. The stream length may accordingly be signalled.

As indicated above, in preferred embodiments the type of symbol to beexpected in the data may be determined by mapping the first type ofsymbols to an ordered tree where the first type of symbols are nodesymbols as described above. The second type of symbols may be datasymbols as described above. In the range encoding example, the datasymbols may not be of a fixed width. It will be understood that theprocess of coupling a de-sparsification algorithm and an entropydecoding algorithm may be applied to other de-sparsification algorithmsor alternatively other algorithms comprising interspersed symbols ofdifferent types.

The process of encoding the bytestream may be substantially mirrored.That is, a sparsification algorithm as described above outputs a symbolto an entropy encoder, optionally together with an indication of thetype of symbol being passed. The entropy encoder then encodes thatsymbol based on parameters or metadata for that type of symbol. The typeof symbol may be determined from the symbol itself or based on anexplicit signal from the sparsification module. The first type of symbolencoded represents an indication of the symbols expected later in thedata set.

As described above, the present invention provides a methodology andapparatus for encoding and decoding image and video data in a mannerthat enables a full flexibility in terms of being able to collect anddetermine statistics associated with video data. Such data providesadvantages in terms of encoding, and decoding, as well as a greaterunderstanding of the data to be encoded. In particular, the presentinvention allows for the identification of tiles with the same orsimilar statistics to be grouped together, even though they may bespatially separate. Thus the overhead in metadata is lowered, whilstallowing for the recordal, and transmittal, of the statistics. Incontrast to prior art systems, the may optionally utilises blocks oftiles of a fixed size (or dimensions) and allows for non-adjacent,spatially separate, tiles to be grouped together. Again, a tile hererefers to a data structure representing an area of the information thatis to be compressed.

FIG. 11 is a block diagram of a system for performing the statisticalcoding methodology.

In FIG. 11, there is shown the system 1100, the system 1100 comprising astreaming server 1102 connected via a network 1114 to a plurality ofclient devices 1130, 1132. The streaming server 1102 comprising anencoder 1104, the encoder configured to receive and encode a first videostream utilising the methodology described herein. The streaming server1104 is configured to deliver an encoded video stream 1106 to aplurality of client devices such as set-top boxes smart TVs,smartphones, tablet computers, laptop computers etc., 1130 and 1132.Each client device 1130 and 1132 is configured to decode and render theencoded video stream 1106. The client devices and streaming server 1104are connected via a network 1114.

For ease of understanding the system 1100 of FIG. 11 is shown withreference to a single streaming server 1102 and two recipient clientdevices 1130, 1132 though in further embodiments the system 1100 maycomprise multiple servers (not shown) and several tens of thousands ofclient devices.

The streaming server 1102 can be any suitable data storage and deliveryserver which is able to deliver encoded data to the client devices overthe network. Streaming servers are known in the art, and use unicastand/or multicast protocols. The streaming server is arranged to encodeand store the encoded data stream, and provide the encoded video data inone or more encoded data streams 1106 to the client devices 1130 and1132. The encoded video stream 1106 is generated by the encoder 1104.The encoder 1104 in FIG. 11 is located on the streaming server 1102,though in further embodiments the encoder 1104 is located elsewhere inthe system 1100. The encoder 1104 generates the encoded video streamutilising the techniques described herein.

The encoder further comprises a statistical module 1108 configured todetermine and calculate statistical properties of the video data.

The client devices 1130 and 1132 are devices known in the art andcomprise the known elements required to receive and decode a videostream such as a processor, communications port and decoder.

The technique takes into account the fact that areas of an image may beidentified by virtue of the fact that they have similar, or identical,statistics. Regions which have the same statistics can, by using Shannonencoding (or any other form of entropy encoding), be described in a lowdata manner. As the entire region is defined in the same manner, thisonly needs to be defined once. It is known in the art to group regionsspatially, on the assumption that pixels which are proximate to eachwill often show similar or identical properties. However, such spatialgrouping will not be particularly effective where strong discontinuitiesare present (for example in the form of a feature or edge). Furthermore,the inventors have beneficially realised that whilst spatial proximateareas will have similar statistics, often areas which are spatiallydistinct will also have similar or identical statistics.

The described concept is based on the idea of identifying areas withinthe image which have similar statistics, and grouping these regions inorder to reduce the amount data required to define such areas byencoding the groups using entropy encoding.

FIG. 12 is an example of an image to be encoded using the method. Theability is provided to group tiles based on statistical properties,wherein the decision regarding the grouping is independent of thelocation of the tile within the image. Accordingly, such grouping allowsfor tiles to be grouped which would not otherwise be grouped based onspatial properties.

In FIG. 12, the image to be encoded is that of a chessboard 1200. As isknown with such boards, the board is defined by a set of alternatingblack and white squares. Three separate and equal sized regions havebeen identified on the board 1202, 1204 and 1206. Whilst the image ofthe chessboard has been used for ease of understanding, the skilledperson would realise that the described concepts are applicable to anyform of image.

In FIG. 12, the first region 1202 defines a region which comprises botha portion of a black and a white square. As can be seen, whilst thepixels in the first region 1202 are spatially proximate, they will havedifferent properties. In particular, the statistical properties of thepixels (for example luma, chroma, number of bits required to define thepixel, etc.) will be different for the black and white pixels within thefirst region 1202. There is also shown region 1202A, which is spatiallyseparate from the first region 1202, but has identical content As suchregion 1202A would have identical statistical attributes to 1202.

Similarly, the second 1204 and third 1206 regions consist of solelyblack and white pixels respectively. The second 1204 and third 1206regions also have further regions 1204A and 1206A which are identical,and therefore will respectively have identical statistical attributes.

The example shown in FIG. 12 is an example where the properties ofspatially distinct regions will be the same, or substantially similar.However, it is found that such clustering is found in many scenes, andthus can be used for the basis for entropy encoding.

Whilst it is possible to encode based on grouping of statistics, theamount of metadata required will also increase when more groups areintroduced. As metadata is generally incompressible, the cost of themetadata must also be considered in order to ensure that the cost of themetadata does not exceed the benefit of the entropy based encoding.

FIG. 13 is a flow chart of the process of clustering data. At stepS1302, a first frame of video data to be encoded is received. The datamay be stored locally, or received from an external source such as aserver. At step S1304, the frame is subdivided into a plurality oftiles. The tiles may be of any suitable size such as 8×8, 16×16, 32×32etc. pixels. The size and number of tiles for the frame may varyaccording to available processing power, and the need to obtain asuitable size of tile in order to obtain meaningful statistics.Preferably, the number and size of tiles is fixed for the frame and allother frames of video data. The step of portioning an image in such amanner is known in the art. Each tile is preferably assigned an indexwhich is indicative of the position of the tile within the image.Preferably the tiles are ordered within the image in a set-order. In apreferred embodiment, the order is set according to a z-order traversalof the tiles, as exemplified in FIG. 14.

In FIG. 14 there is shown an image 1400 which is subdivided into aplurality of tiles 1402, each tile containing a number of pixels 1404.Each tile 1402 is assigned an index 1408 found according to the z-ordertraversal 1406 which preserves the spatial proximity of the tiles.Efficiently computing the z-order traversal 1406 is known in the art.Thus, there is defined a set of indices which define the tiles. As theorder is set, and the size of the tiles consistent, the individualpixels which define a tile can be readily determined.

Returning to FIG. 13, at step S1306, for each of the tiles identified atstep S1304, the statistical attributes for the tile are calculated.

In an embodiment, the tile is of dimension 16×16 pixels and thereforecontains 256 total pixels. This allows for meaningful statistics to becalculated. The statistical attributes of the tile define one or moreproperties of the tile such as luma, chroma, number of bits required todefine the tile, etc. An aspect is that the statistical attributes areobjectively measurable, thus allowing for an objective comparison of thestatistics between tiles to be made. The measurement of statistics for atile is known in the art, and occurs via known means.

Therefore at step S1306 for each tile one or more properties/statisticalattributes for each of the tiles are identified.

However, in an embodiment whilst such properties or statistics arecalculated, they are not stored with the source data stream as metadata.They are stored separately from the data stream; for example, in memoryassociated with a processor performing the encoding.

At step S1308, instances of tiles defining the same or similar instancesof a statistical attribute are identified. The identified tiles aregrouped together so as to define a tileset. A tileset is therefore agroup of tiles (which may, or may not, be spatial connected) that aregrouped on the basis of having similar statistical attributes. As thetiles within a tileset have similar, or identical attributes, they willbe near identical (or identical).

By grouping similar tiles into tilesets, the amount of metadata neededcan be reduced. FIG. 15 is an example image 1500 that has beensubdivided into 9 tiles. Without the use of tilesets, metadatacorresponding to 9 separate tiles would need to be used. However, sometiles share similar statistic properties with other tiles, such as 1502Aand 1502B. Therefore, similar tiles are grouped together to formtilesets, exemplar tiles for each tileset being 1502A, 1504, and 1506.As such, metadata corresponding to three tilesets would need to be usedleading to a more compact metadata representation of the image 1500.Thus, an important aspect is the clustering of tiles into tilesetsaccording to their statistical attributes or properties, S1308.

As each group has similar or identical statistical attributes, the groupcan be defined by their statistical attributes and are identified astile sets.

In an embodiment, tilesets are formed according to the cumulativeprobability distribution function of a particular statistic or set ofstatistics computed over the tiles. An example of a cumulativeprobability distribution function is shown in FIG. 18.

FIG. 18 is a representative example of a statistic plotted against itscumulative probability to form a cumulative probability distribution1802. Whilst the general shape (the elongated s-shape) of thedistribution function 1802 will be the same across different frames, thespecific form of the distribution will be dependent on the values forthe frame. As the values for each frame change, the specific shape ofthe distribution function 1802 will change. Thus, having tilesetsdefined on the distribution function provides a more effective measureacross a frame. The use of a distribution function furthermore allowsfor tiles with similar statistics to be easily identified. In apreferred embodiment, sixteen groups of tiles (i.e. tilesets) areidentified. The tilesets are identified in this instance by sampling 17points across the cumulative probability distribution function 1802.These points then form the boundaries for each tileset. 1804A and 18048define the boundaries for one such tileset whereby any tile that has astatistic value, s, in the range, a<s≤b, is assigned to that tileset. Asthe statistics and their distribution will change for each frame, theboundary values will also change between frames. Whilst sixteen tilesetsis preferred, in further embodiments other numbers of tilesets may beused.

In further embodiments, other forms of ordering of statistics, andclustering of tiles based on the similarity of statistics, are used. Aseach group has similar or identical statistical attributes, the groupcan be defined by their statistical attributes and are identified astile sets.

A key aspect is that the identification of the individual tiles whichform the tile set is independent of the spatial distribution of thetiles. Therefore tiles which are spatially distinct may form part of thesame group. Thus at step S1308 the identity of the titles forming eachtileset is recorded.

At step S1310 metadata for each identified tileset is calculated. Themetadata describes the properties of the tiles which form the tileset,preferably comprising the data to describe one or more statisticalattributes of the tileset, and the identification of the tiles whichform the tileset. In one embodiment, the statistics for the entiretileset are recalculated using all tiles within the tileset. In furtherembodiments, the statistics for the tileset are taken from a single tile(for example the median tile of the tileset) and applied across thewhole tileset. Thus, the metadata defines one or more statisticalattributes for the tiles, as well as the individual tiles which form thetileset.

The process at step S1310 is repeated for each of the tilesets. Thus, aswell as the identity of the tiles forming the tileset (as determined atstep S1308) the properties of the tiles—which are identical orsimilar—are also determined. Such data is stored in the form of metadataassociated with the image and tileset.

As the tiles are encoded in a sequential manner, the data or metadata isin the form of a set of indices which define the statistical attributesby which the tilesets are defined. These indices may be costly in termsof overhead. The inventors have beneficially realised that by clusteringthe tiles into tilesets, via their statistics, a measure of thelikelihood of the data having a particular value (i.e. the value of thestatistics by which the tileset is defined) can be made. Thus at stepS1310, in an embodiment, entropy encoding is used to reduce the costassociated with encoding. As the tilesets are defined by multiple tiles,the overhead associated with the metadata for entropy encoding is lessthan the cost of entropy encoding thereby producing a reduction in datasize.

At step S1312 for each of the tilesets, the tileset and metadatadescribing the tileset are encoded.

As the tilesets are identified by their similar, or identical,statistical attributes, the metadata to describe the statisticalproperties of the tiles in the tileset is constant for all tiles withinthe tileset. Thus, as the tiles within the tileset share the sameproperties, the data requirement in order to encode the frame is greatlyreduced as the metadata required to describe each tile in the frame inan embodiment is entropy encoded.

Whilst the above process allows for reductions in the dataset to be madebased on the statistical similarity of the tiles in the dataset, furthergains may be made by taking advantage of the similarity of tiles whichare neighbouring each other. As is known, tiles which are adjacent toeach other often show little or no variation. Therefore, in order toreduce the amount of data required to encode the tiles, the tiles arepreferably encoded in a set order, such as a z-order traversal as shownin FIG. 14. Preferably, the value encoded is the difference between thevalue for the tile and the preceding value. Thus if the two tiles areidentical in value then the encoded value is zero, or if they aresimilar then the value is small. Both of which have a low bit costassociated with their encoding. As stated above, tiles which arespatially similar (i.e. near to each other) will also typically havesimilar statistics. Therefore, whilst the invention will beneficiallygroup the tiles in the tileset without any consideration to the spatialsimilarity, by ordering the encoding of the tiles in a z-ordertraversal, the encoding process can beneficially use the advantagesassociated with both the spatial and statistical similarities during theencoding process.

A further advantage associated with the use of tilesets is that theyallow for decisions to be made at statistical level across the entireimage. It is known in video encoding to encode different frames atdifferent levels of encoding as part of an adaptive encodingmethodology. The decision is made based on the frame level statisticswhich have been determined. A further advantage of the present inventionis that the use of tilesets, and the metadata associated with thetilesets (which is applicable to all the tiles that form the tileset)allows for decisions regarding adaptive encoding to be made on aframe-by-frame basis, with the variation in the encoding occurringwithin the individual frames.

FIG. 16 is a flowchart of an adaptive encoding process within a frame.

As is known in video encoding, the bandwidth available is a limitingfactor to the amount of data that can be sent and ultimately the levelof quantisation used when encoding. There are a number of objectivequality of frame metrics which are used to provide an objective measureto the encoded picture quality when compared to the original sourcedata. Such measures are made across the entire frame. There are also anumber of subjective measures, which define how well the end user willperceive the image. As the above described methodology allows for thelocal statistics to be obtained, it is possible to use such informationin order to identify areas within a frame which are likely to be seen asvisually more important to the end user. Such areas can be selectivelyencoded at a higher level of quantisation in order to provide a higherquality image for those areas. However, due to the finite amount ofbandwidth, a trade-off must be made and the level of quantisation inother areas of the image must be lowered. As such the objective imagequality metric for the entire image remains constant, but the areaswhich are deemed to be visually more important are quantised at a higherlevel. This therefore helps to provide a subjectively improved image.

At step S1602, the process begins with the receipt of the metadata foreach tileset as determined at step S1310.

At step S1604, the metadata is used to determine tilesets which arelikely to contain edges, or feature information, in the image. Features,such as edges, are known in video encoding to be a source of compressionartifacts. Such features may be identified by the statistics of theframe, with certain statistics being associated with encoding errors. Ascompression artifacts are visible to the user, tiles which havecompression artifacts will have, on a user subjective level, a lowerlevel of quality.

In an embodiment, the tilesets are ranked by a statistical value todefine an order of visual “importance”. In an embodiment, the statisticsrelate to the error associated with the encoding process and thetilesets are ranked from smallest to largest error. Tilesets which areidentified as having the largest error are typically those associatedwith edges, or feature information, and will be perceived by the enduser as having the lowest level of quality. In further embodiments,other suitable methods of ranking the tilesets are used.

At step S1606, the metadata is used to determine tilesets which arelikely to be featureless, containing a uniform colour with little or novariation. Such tiles may be associated with a uniform backgroundfeature, or a consistent feature. Such tiles are also identified by thestatistics as they show no variation in values across the tile. As suchfeatures are constant that are associated with having no compressionartifacts or the like.

At step S1608, an adaptive quantisation decision is made in order todetermine what level of quantisation is used to encode the individualtilesets. As a default, all tiles are encoded at the same level ofquantisation. This is the standard encoding behaviour where the entireimage is encoded at the same level of quantisation.

It has been beneficially realised that the level of quantisation acrosstilesets may be varied, with certain tilesets being encoded at a higherlevel of quantisation than others. Such decisions can be made asstatistics are provided at an individual tileset level, therebyproviding the information required to make the decision. Beneficially,by encoding tilesets, and therefore the tiles, identified at step S1604(i.e. those which are likely to be associated with compressionartifacts), such tiles will show fewer compression artifacts andtherefore will be perceived by the end user/viewer of the video to be ofa higher quality. However, as the bandwidth cannot be increased, acorresponding decrease in the level of quantisation is made for some orall of the tilesets identified at step S1606. As the level ofquantisation for these tilesets has decreased, fewer bits are requiredin order to encode such frames. Thus, the overall amount of datarequired to encode the frame remains the same, but the level ofquantisation is varied across the frame in order to provide enhancedregions (and corresponding regions with lower quantisation) within theimage. Therefore, the subjective measure for the image will increase asthe regions which are likely to show compression artifacts are quantisedat a higher level, whereas areas which are uniform can be encoded at alower level of quantisation without adversely affecting the end userexperience.

In an embodiment at step S1608, a first tileset is selected to beencoded at a higher level of quantisation. Preferably the tileset to beencoded at the higher level of quantisation is the tileset which isranked as the visually most important tileset. Subsequently, theincrease in the size of the encoded frame as a result of encoding thetileset at the higher level of quantisation is determined. In order toensure that the encoded dataset does not exceed the available bandwidth,one or more further tilesets are identified to be encoded at a lowerlevel of quantisation. Preferably, the identified tilesets are thosewhich are deemed to be visually the least important. \Mien the tilesetsare encoded at the lower level of quantisation, the decrease in the sizeof the encoded frame is determined and compared with the increase as aresult of the quantisation of the tilesets at the higher level ofquantisation. This process is repeated until such time that the overallsize of the frame is the same, thus ensuring that the bandwidthrequirement is not increased. Accordingly, the process provides animproved encoding process.

The process of creating tilesets can be repeated for an individual framemultiple times, with each set of tilesets determined based on aparticular statistical attribute. By having multiple sets, furtherinformation regarding the underlying frame of video data is recorded andcan be beneficially used in the encoding and decoding process. However,as there is an overhead associated with the creation of the set, inpractice it is not desirable to have multiple sets.

A further aspect of the invention is the ability to group tiles intotilesets based on multiple statistical attributes. FIG. 7 is a flowchartof the process of defining a set of tilesets based on multiplestatistical attributes.

Steps S1702 and S1704 are identical to steps S1302 and S1304respectively.

Whilst it is possible to define and record the data for each statisticalattribute separately this is costly in terms of the size of datarequired. Beneficially, the above methodology can be used to group tilesto form a tileset using several statistics.

In order to group the statistics, at step S1706 each statistic isdefined in terms of a vector and a single vector score is determined forthe statistics. It is found that clustering of up to three differentstatistics is preferred though in further embodiments a different numberof statistics are grouped.

Therefore, following the same principles described above with respect tostep S1308 of FIG. 13, at step S1708 the tiles are clustered in groupsof tiles having the same or similar vector (and thus same or similarstatistics). As described with reference to FIG. 13, such clustering ofthe tiles allows for the tilesets to be encoded and further allowing forentropy encoding.

In further embodiments the value of each statistic is stored in a singlematrix and the clustering is based on instances of the same of similarstatistics.

The following sets out features of the hierarchical data structure usedin accordance with certain embodiments and associated advantages of thathierarchical data structure.

There is provided a method of decoding a received set of encoded datarepresenting information that has been compressed, wherein the encodeddata set is divided into a hierarchy of subsets, the method comprising:decoding at least one first subset to delve a respective set ofattribute metadata elements; separately decoding a plurality of secondsubsets comprising data elements, wherein each second subset describes aregion of the information that has been compressed; and, reconstructingthe information that has been compressed from the data elements, whereinthe region of the information that has been compressed is identifiedfrom the attribute metadata elements.

It is provided that individual portions of a bytestream can be decodedwithout reference to other similar sections of the same bytestream. Thisfacilitates increased memory utilisation as each portion, onceseparately decoded, can be spatially located in the information beingcompressed without having any knowledge of the other similar sectionsand without additional information being stored in memory. The data inmemory at an instant may be a subset of the data of a whole plane.Conventionally, an entire plane is decoded as one such that it cannot bebroken to enable separate decoding. By separately we consider that thefirst subset may be separately decoded from the second subsets, thesecond subsets may be decoded separately from other, or preferably both.That is, that each subset is decoded separately from any other subset.

Each second subset may comprise a data structure of structure metadataelements and data elements and reconstructing the information that hasbeen compressed from the data elements may comprise spatially arrangingthe data elements in an array based on the structure metadata elements.Thus each of the second subsets may themselves be a data structure. Inthis way the proposed technique can be thought of as the breaking downand reconstruction of a larger data structure with each part of the datastructure being separately decodable so that data stored within eachstructure can be spatially located without knowledge of the other partsof the data set.

Reconstructing the information that has been compressed may comprise:inserting a predetermined series of data values in regions of the arraywhere the attribute metadata elements indicate that no second subset isincluded in the dataset for a region of the array. Regions of theinformation which have consistent values need not be signalledexplicitly in the bytestream and therefore the overall size of the datacan be reduced. This may be thought of as sparsifying the data. Thehierarchical data structure provides a mechanism by which a regionimplicitly signalled in the data set can be accurately located and theinformation contained therein populated without that region beingincluded within the bytestream or decoded at the decoder. Not only doesthis reduce data size but it also dramatically increases decoding speedas swathes of subsets do not need to be decoded for largely consistentdata arrays, such as residuals.

The attribute metadata elements may indicate the predetermined series ofdata values for a respective one of the plurality of second subsets. Itthus becomes possible to signal the large swathes of information abovedifferently where there are different consistent regions within a planeor between planes. Alternatively, the predetermined series of datavalues may be known to the decoder and may be zero values.

The attribute metadata elements may comprise a flag indicating that nosecond subset is included in the dataset for a region of the array. Theflag facilitates the decoder identifying that a subset is not expectedin the dataset for that region. Where the first subsets correspond to afirst tier of the hierarchical data structure and the second subsetscorrespond to a second tier of the hierarchical data structure, the flagmay indicate if a corresponding data structure exists in a subsequenttier.

Each of the attribute metadata elements may describe a respective one ofthe plurality of second subsets. For example, the attribute metadataelements may indicate specific attributes or parameters of a specificone of the second subsets. In this way, one first subset may describe aplurality of second subsets. Where the first subsets correspond to afirst tier of the hierarchical data structure and the second subsetscorrespond to a second tier of the hierarchical data structure, the datastructure may be visually represented as an inverted pyramid or tree.Each attribute metadata element may accordingly correspond to a sub-gridof an overall-grid to be decoded.

In general, the technique proposes the concept of separately decodingdata structures using information contained within a different datastructure.

The attribute metadata elements may comprise the dimensions of a datastructure in a respective one of the plurality of second subsets. Thuswhen decoding the second subsets and placing the subsets in the array,the decoder is able to identify the expected shape and size of the datafor placing in the region. Alternatively, the dimensions may besignalled separately or may be of a fixed size for each tier of thehierarchy.

The attribute metadata elements may comprise location information toenable the decoder to locate a respective one of the plurality of secondsubsets, and the step of separately decoding a plurality of secondsubsets may further comprise: searching for at least one of theplurality of second subsets based on the location information. Theattribute metadata elements thus provide for the parallel decoding ofthe second subsets and the random access of those subsets such that eachdoes not need to be decoded to accurately recreate portions of theinformation that has been compressed. The location information may befor example lengths of subsets, an offset from a location in the datasetor a fixed location in the data set.

The attribute metadata elements may indicate decoding parameters. Thedecoding parameters may be used by the decoder to differentially decodethe second subsets from one another and thus improve the efficiency ofthe overall decoding process as parameters can be optimised for eachsubset. The decoding parameters may for example be entropy decodingparameters such as statistics or an indication of statistics to use fordecoding a subset. Additionally, parameters signalled by the attributeelements may be quantization parameters.

The plurality of second subsets may be decoded based on the attributemetadata elements. Where the attribute metadata elements indicatedecoding parameters the decoding may be performed according to thoseparameters to improve decoding efficiency. Where the attribute metadataelements include for example a length of the second subset or dimensionsof the data structure, the decoder may be tailored to that particularsubset to improve overall efficiency. In the art, typically the entiretyof the data elements or the entirety of a graph is decoded together andis not decoded separately and therefore such benefits of decoding eachsubset differently cannot be realised.

The method may further comprise: mapping the attribute metadata elementsto a first tier of a hierarchical data structure; and, mapping eachdecoded second subset to a second tier of the hierarchical datastructure. The mapping of each subset to a tier in a data structurefacilitates the implementation of the spatial location of the subsets.Once the information has been decoded it can be placed in the datastructure. The data structure may be a linear data structure or anon-linear data structure such as a graph. This technique allows a datastructure to be broken into sections which can be decoded separately andthe data structure to be subsequently recreated at the decoder withoutexplicitly signalling the data structure. That is, an unbroken graph maybe broken into a series of graphs or separate data structures which canbe separately decoded. Optionally, the decoded second subset may begrated to the decoded attribute metadata elements. For example, wherethe first and second subsets are a each portions of a graph, metadata ofa second subset may replace leafs of first subset graph to recreate anunbroken graph.

The method may further comprise: mapping each data element of the secondsubsets to an array based on its location in the data structure. In thisway the spatial location of the original data elements in the originaldata structure is maintained where the data structure reflects an arrayof information. The mapping of the data to a data structure helpsidentify the spatial location without storing the data structure inmemory.

Each decoded second subset may be mapped to the hierarchical datastructure based on the attribute metadata elements. For example, themethod may comprise mapping each decoded second subset to a location inthe second tier of the hierarchical data structure based on theattribute metadata elements. The attribute metadata elements mayindicate that a location of the second tier of the data structure doesnot correspond to a second subset.

The mappings may be performed according to a predetermined order, forexample a Morton or z-order. Alternatively the ordering of the secondsubsets may be indicated in the attribute metadata elements. The spatialinformation of the data set or data structure may be varied by explicitsignalling in the attribute metadata elements.

The encoded data set may further comprise a plurality of first subsets,and the method may further comprise mapping a subset of the plurality offirst subsets to a root tier of the hierarchical data structure; mappinganother subset of the plurality of the first subsets to an intermediatetier of the hierarchical data structure, wherein the attribute metadataelements of the root tier describe the first subset of the intermediatetier. In this way the hierarchy may be increased to multiple tiers suchthat a large amount of data can be encoded and decoded in manageableportions. The process may be recursed for multiple tiers.

The first subset may be a data structure comprising structure metadatawhich indicate that no attribute metadata element is included in thefirst subset for a location in the data structure and that acorresponding location of the second tier does not correspond to asecond subset. In this way the overall size of each subset may bereduced and the decoder may be able to easily identify that a regions ofthe array should be recreated as having consistent values without asuitable attribute metadata element being explicitly included in thesubset.

Preferably the data elements may be data symbols and the set ofattribute metadata elements may be in the form of a tuple.

Further, there is provided a method of processing metadata associatedwith a stream of video data, the method comprising the steps of, for afirst frame of video data: subdividing the first frame of video datainto a plurality of tiles; calculating a first statistical attribute foreach of a plurality of tiles; identifying tiles having a first instanceof identical, or similar, statistical attributes; grouping saididentified tiles together as a tile set; for each tile set definingmetadata for the tile set, said metadata indicative of the statisticalattribute of the tiles defining the tile set; and encoding dataindicative of the metadata for the tiles of the first frame of videodata based on the metadata defined for each of the tile set to whichsaid tile belongs.

Thus, the method provides the means for identifying and grouping tileswhich are not necessarily linked spatially, but are linked by theirstatistical properties. Being able to group tiles in such a mannerprovides an improved understanding of the properties of the data, andfurthermore allows for groupings to be made which would not otherwise bemade.

Optionally, the encoding occurs using an entropy encoding basedtechnique. As the groupings are based on their statistical similarity,the probability of the occurrence of the data can be calculated allowingfor entropy based encoding.

Optionally, the metadata for a tile set further defines a location ofeach of the tiles forming the tile set, and preferably wherein the sizeof the tiles is fixed. This allows for an improved understanding of thedata which can be repeated across multiple frames or datasets.

Optionally, wherein the step of identifying the tiles that form a tileset comprises further comprising ordering the tiles, based on theirstatistical attributes, preferably wherein the ordering of the tilesdefines a probability distribution function of the statisticalattributes. Such ordering enables the easy identification of tiles whichhave identical or similar attributes.

Optionally, wherein tiles are encoded in a set order, preferably whereinthe method further comprises determining the difference between themetadata of a tile and its preceding tile, and encoding the metadata asthe difference, preferably wherein in the set order is a z-ordertraverse. This allows for further reductions in data to be made.

Optionally, wherein the method further comprise the step of encoding thetiles, preferably wherein the step of encoding the tiles comprisesdetermining a level of quantisation and encoding tiles at said level ofquantisation, more preferably wherein tiles belonging to a first a firsttile set are quantised at a first level of quantisation and tilesbelonging to a second tile set are quantised at a second, different,level of quantisation. The improved understanding of the video dataprovided by the clustering based on the statistics allows for areas ofthe image to be selectively quantised at higher rates of quantisation.Thus specific areas within the image which are deemed to be importantcan be encoded at a higher level of quality thus producing an improvedimage.

It has been described above how attributes of a later data structure maybe signalled in the bytestream. Specifically, it is described that ahierarchical data structure may be provided. The following describes howa decoding module may obtain a value from a bytestream or data set anddetermine based on this value a type of shortcut used in the encodingprocess and/or to be used in the decoding process. The decoding modulemay configure the decoding process to adapt its operations based on theindicated shortcut and/or implement a decoding process which was theindicated shortcut. The specific types of shortcuts, what each typemeans and what advantages it provides may be described below.

In certain examples, the value may be included in a header of abytestream and may indicate one or more modifying operations used in theencoding process when building a header and/or to be used in thedecoding process in order to decode a bytestream. These modifyingoperations, or shortcuts, provide many general advantages such as toreduce the amount of data to be encoded/decoded and/or to optimise theexecution time at the decoder, for example by optimising the processingof the bytestream.

The terminology used herein may describe these shortcuts astransformation rules, variation parameters or adaptation parameterswhich indicate modifying operations or a modified decoding process ordecoding module to be used. Such terminology may be used interchangeablyto refer to similar functionality.

In certain examples, the decision to implement these operations may be arate-control decision.

As noted above, the shortcut may be implemented as a string of bits in aheader of a bytestream or bitstream. Thus, depending on theconfiguration of the header and payload, the modifying operator may beapplied to the whole of a plane of data, multiple planes or to aparticular level of quality (LOQ) or set of data. It is of coursecontemplated that the shortcut value may be signalled in any othersuitable means to modify operation of the decoding module.

The choice of shortcut may for example be constant for a version of aplane and the shortcut is fixed. For a new plane or version of a plane,one can change the shortcut.

When encoding a plane, the encoder can consider which modifyingoperation or option will be more beneficial. For example, an extra tablewill take up space in the stream but can save overall space. If thereare 1000 tuples all having a number 5 or less then a list may need to be5 long but supplies an attribute to a 1000 different subsets. In thisway, the shortcut can indicate how the data is set or retrieved at thedecoder.

In a particular example, this concept may have particular utility withother aspects of the present disclosure. For example, in certainembodiments the shortcuts control which attributes are associated withthe data structures. Similarly, the shortcuts can indicate whichimplementation option is shown from a plurality of implementationoptions.

Examples of several possible shortcuts will now be described.

In one example, the shortcut may indicate that where there is a list ofattributes, a single index value may indicate which of a set ofattributes are to be used for each subset. That is, where a tableau datastructure or root data structure indicates the attributes of a laterdata structure, an index value may point to a set of attributes to beused.

Alternatively, the index value may instead point to a set of furtherindices. The shortcut may indicate which of these options is to be used.

In a specific example, an index of 33 in the data structure may indicatethat a later data structure should be decoded according to a set ofattributes pointed to by 33 in a list. A different shortcut may indicatethat instead 33 points to a list of tuples e.g. {43, 18, 5}, i.e. row 43in list A, row 18 in list B and row 5 in list B. Thus, the shortcut canoptimist the communication of index values to the decoder from theencoder and the speed of decoding the data, together with data overhead.

Of course it will be understood that these attributes and tuples may bestored in a data store or signalled in the bytestream or bitstream.

In a further example, a shortcut may indicate that an index is not sentor an attribute is not indicated in the data structures. That is, theshortcut may indicate that some data structures are decoded according todifferent attributes and some do not. In a detailed implementation ofthis example, data structures of a different plane are not allowed tohave different attributes but between planes they can be different,possibly without those being signalled in the header. This can besummarised as an intra-plane or inter-plane difference.

In an additional shortcut example, the shortcut may indicate to thedecoder that no node signals are sent. For example, in the decodingprocess described above, all node signals can be assumed to be [1111]and thus cost nothing to send. The data structure may only contain dataelements but these can be spatially located using the predeterminedmapping orders described above. In sum, all quadtrees may be dense, butthe symbols are not sent to reduce overhead.

In a particularly advantageous example of a shortcut, the encoder mayindicate to the decoder that no quantization parameters are sent andquantization has not occurred. Thus the decoder may not implementquantization for the data structure. This provides for selectivelossless encoding. In sum, the shortcut may indicate that quantizationis disabled.

Similarly, a shortcut may indicate a particular transformation should beperformed on a block.

Thus it can be seen how a shortcut value or variation parameter can beused by the decoder to vary a decoding operation or utilise a differentdecoding operation to optimise decoding of a set of data.

It will be clear to one skilled in the art how techniques describedherein may be embodied within a system comprising an encoder and adecoder. At the decoder, the encoded data set may be retrieved from adata store or received from a streaming server. In such a furtherembodiment, one or more streaming server(s) may be connected to aplurality of client devices. At the streaming server, the encoder mayreceive and encode a video or image stream and deliver the stream (e.g.bytestream or bitstream) to the client devices. Thus the stream can bedecoded by a decoder to recreate the information that has beencomprised. Any suitable mechanism to deliver the stream may be used,such as unicast or multicast, as would be well-known to the skilledperson.

Techniques described here may be suitable for the encoding, decoding andreconstruction of any dimension array of data. However, although thetechniques are also applicable to linear data, they are most beneficialfor image or video reconstruction. In the case of a picture or video,the data could be values associated with a colour space (e.g., the valueof a red component in an RGB colour space, or the value of a Y componentin a YUV colour space, etc.), or alternatively the data could beresidual data (whether transformed or not) or metadata used to decode abytestream or bitstream. Residuals are further defined in the presentapplication, but in general residuals refer to a difference between avalue of a reference array and an actual array of data. Thus, thetechniques are most suitable for any plane of data.

It should be noted that techniques described in the following examplesare agnostic as to the meaning or use of the decoded array. Of course,the data set may be used to reconstruct a larger dataset by combiningmultiple decoded data sets. Once recreated the data may represent anyinformation which has been compressed, such as an image or sonogram. Aswill be understood from the following described examples, encoding anddecoding techniques wherein a quantity of data to be compressed andtransmitted or stored by way of a scheme involving encoding the data ina hierarchy of data structures from which the original data can bereconstructed are especially suitable for use with the invention.

At both the encoder and decoder, for example implemented in a streamingserver or client device or client device decoding from a data store,methods and processes described herein can be embodied as code (e.g.,software code) and/or data. The encoder and decoder may be implementedin hardware or software as is well-known in the art of data compression.For example, hardware acceleration using a specifically programed GPU ora specifically designed FPGA may provide certain efficiencies. Forcompleteness, such code and data can be stored on one or morecomputer-readable media, which may include any device or medium that canstore code and/or data for use by a computer system. When a computersystem reads and executes the code and/or data stored on acomputer-readable medium, the computer system performs the methods andprocesses embodied as data structures and code stored within thecomputer-readable storage medium. In certain embodiments, one or more ofthe steps of the methods and processes described herein can be performedby a processor (e.g., a processor of a computer system or data storagesystem).

Generally, any of the functionality described in this text orillustrated in the figures can be implemented using software, firmware(e.g., fixed logic circuitry), programmable or nonprogrammable hardware,or a combination of these implementations. The terms “component” or“function” as used herein generally represents software, firmware,hardware or a combination of these. For instance, in the case of asoftware implementation, the terms “component” or “function” may referto program code that performs specified tasks when executed on aprocessing device or devices. The illustrated separation of componentsand functions into distinct units may reflect any actual or conceptualphysical grouping and allocation of such software and/or hardware andtasks.

A technique for decoding a bytestream will now be described.

A decoding module would receive a portion of data to be decoded (e.g.,Stream). This portion of data would be part of a data stream, such as aBytestream or Bitstream. This portion of data may be of variable length(for example, 3 bytes or equivalently 24 bits) and is typicallyassociated with an elementary data structure that describes the data tobe decoded, for example the data structure called Tile as furtherdescribed in the present application and other applications by the sameapplicant such as European patent application No. 17386045.3 and/or17386046.1 both filed on 6 Dec. 2017 and incorporated herein byreference.

To enable decoding of the portion of data, use of some additional datasuch as metadata may be required. This metadata may be present in theportion of data itself (for example, the portion of data may include aheader field containing said metadata and a payload field containingdata to be decoded), or could be received as part of a separate datafield, such as a data field including metadata for multiple portions ofdata (e.g., for all the Streams in a Surface, wherein Surface isdescribed elsewhere) with the portions of data included in a payloadfield. This separate data field may be received prior to the portion ofdata. The header field of the portion of data may be decoded ahead ofthe payload field in order to enable decoding of the data to be decoded.This separate data field may be decoded ahead of a portion of data. Themetadata themselves may be associated with the elementary data structurethat describes the metadata, for example the data structure calledTableau as further described in the present application and otherapplications such as the above-mentioned European patent application No.17386045.3 and/or 17386046.1.

Note that Tile and Tableau are two embodiments of the same datastructure called Tessera, as further described in the presentapplication and other applications by the same applicant such as theabove-mentioned European patent application No. 17386045.3 and/or17386046.1.

As discussed above, the data stream (e.g., Bytestream) may includemultiple portions of data. Typically, there are no gaps betweendifferent portions of data—in other words, the last byte (or bit) of afirst portion of data is followed in the data stream by the first byte(or bit) of a second portion of data. The metadata may be used toindicate a length associated with a portion of data (e.g., aStreamLength). These lengths can range from zero to an arbitrary maximumnumber of bytes associated with a portion of stream.

During encoding, the data to be encoded (for example, transformedresidual data) are processed so that they are divided into groupings ofdata, with each grouping of data associated with an elementary datastructure (e.g., Tessera) as discussed above. For example, withreference to FIG. 19, a first grouping of data G1 may be associated witha first elementary data structure T1 and gets encoded as first encodeddata set E1, a second grouping of data may be associated with a secondelementary data structure T2 and gets encoded as second encoded data setE2, a third grouping of data G3 may be associated with a thirdelementary data structure T3 and gets encoded as first encoded data setE1, and so forth. When transmitting to the decoder, a data stream wouldneed to be created, said data stream being formed by a sequence of bytescorresponding to the sequence of encoded data sets, first E1, then E2,then E3 and so forth.

Since the data to be encoded may be sparse in nature (e.g., many ofthose data to be encoded are either zero or below a certain threshold),some of these groupings of data to be encoded may be completely empty,for example G2 may be completely empty. That means that whilst G1 and G3contains some data to be decoded and therefore the corresponding encodeddata sets E1 and E3, respectively, contains data to be decoded, G2 doesnot contains any data and therefore the corresponding encoded data setE2 contains no data. Accordingly, the data stream will contain a firstportion of data corresponding to E1 and a second portion of datacorresponding to E3, with no portion of data corresponding to E2.

Since the decoding module would not know a priori that there is noportion of data corresponding to E2, and since the data stream asdiscussed above has no gaps, the decoder needs to receive informationabout the length of each of the portion of data to reconstruct anddecode the various groupings of data. Accordingly, the metadata MD willcontain information about the length of the various portions of data inthe data stream. In the exemplary FIG. 19, E1 has length of X bytes, E2has a length of 0 bytes, E3 has a length of Y bytes.

The decoding module will extract the length information from themetadata MD, and based on it extract from the data stream thecorresponding portions of data. With reference to the exemplary FIG. 19,the decoding module extracts the length of E1 as X bytes, Accordingly,the first X bytes of the payload data will be associated with E1.Further, since the decoding module would extract the length of E2 as 0bytes whilst the length of E3 as Y bytes, the decoding module willassociate the next X bytes in the payload data with E3, thereforeknowing that E2 has no data associated with it. Accordingly, thedecoding module will decode E1 and E3 to arrive at the reconstructedversions of, respectively, groupings of data G1 and grouping of data G3,but it will not reconstruct any grouping of data G2.

As described in the present application and other applications such asthe above-mentioned European patent application No. 17386045.3 and/or17386046.1, the data to be decoded are organised in tiers of Tesserae,with the top Tier (Tier 0) being the Tesserae associated withtransformed residual data (also known as Tiles), Tier-1 being theTesseare associated with metadata of the Tiles on Tier 0 (these Tesseraealso known as Tableaux), Tier-2 being the Tesserae associated withmetadata of the Tableaux of Tier-1, and so on and so forth. Thesemetadata could be, for example, the length of the portions of dataassociated with the Tiles (if we are referring to Tier-1) or the lengthof the portions of data associated with the Tableaux (if we arereferring to Tier-2).

Accordingly, when a decoding module receives the data stream it shallextract information about the length of the portions of data associatedwith the various Tesserae.

Tesserae are decoded in phases, each phase corresponding to decoding aTier. This is further described in the present patent application. ATableau tier decoding phase involves using Streamlengths to “find” theTableaux for that Tier, then decoding the “found” Tesserae to obtainmore Streamlengths. The Tile tier decoding phase involves usingStreamlengths to find the Tiles, and decoding the “found” Tiles to getresiduals (all other residuals being zero).

As shown in FIG. 23, the bytestream may include multiple fields, namelyone or more headers and a payload. In general, a payload includes theactual data to be decoded, whilst the headers provide information neededwhen decoding the payload. The payload may include information about aplurality of planes. In other words, the payload is subdivided inportions, each portion corresponding to a plane. Each plane furthercomprises multiple sub-portions, each sub-portion associated with alevel of quality. The logical structure of a Payload is an array ofmulti-tiered Tableaux, which precedes the Tile Tier with Tilescontaining Residuals at their Top Layer. The data in the Payload thatrepresents a Tessera shall be a Stream. In the present example, Streamsare ordered by LoQ, then by Plane, then by direction and then by Tier.However, the Streams can be ordered in any other way, for example firstdirection, then LoQ, the Plane, then Tier. The order between directions,LoQ and Planes can be done in any way, and the actual order can beinferred by using the information in the header, for example the streamoffsets info.

The payload contains a series of streams, each stream corresponding toan encoded tessera. For the purpose of this example, we assume that thesize of a tessera is 16×16. First, the decoding module would derive aroot tableau (for example, associated with a first direction of a firstLoQ within a first plane). From the root tableau, the decoding modulewould derive up to 256 attributes associated with the corresponding upto 256 tesserae associated with it and which lie in the tier above theroot tier (first tier). In particular, one of the attributes is thelength of the stream associated with the tessera. By using saidstreamlengths, the decoding module can identify the individual streamsand, if implemented, decode each stream independently. Then, thedecoding module would derive, from each of said tessera, attributesassociated with the 256 tesserae in the tier above (second tier). One ofthese attributes is the length of the stream associated with thetessera. By using said streamlengths, the decoding module can identifythe individual streams and, if implemented, decode each streamindependently. The process will continue until the top tier is reached.Once the top tier has been reached, the next stream in the bytestreamwould correspond to a second root tableau (for example, associated witha second direction of a first LoQ within a first plane), and the processwould continue in the same way.

The bytestream may include a fixed-sized header, i.e. a header whosebyte/bit length is fixed. The header may include a plurality of fields.FIG. 20 shows an example of said fixed-sized header.

The fixed-sized header may include a first field indicating a version ofthe bytestream format (B.1—also described as format_version: unit8). Inan embodiment, this first field may include 8 bits (or equivalently 1byte). This field may allow flexibility in the encoding/decoding processto use, adapt and/or modify the version of the bytestream format andinform a decoding module of said version. In this way, it is possible touse multiple different version of the encoding/decoding format and allowthe decoding module to determine the correct version to be used.

A decoding module would obtain said first field from the bytestream anddetermine, based on the value included in said first field, a version ofthe encoding format to be used in the decoding process of saidbytestream. The decoding module may use and/or implement a decodingprocess to adapt to said version.

The fixed-sized header may include a second field indicating a size ofthe picture frame encoded with a specific bytestream (B.2—also describedas picture_size: unit32). The size of the picture frame may actuallycorrespond to the size of the bytestream associated with that pictureframe. In an embodiment, this first field may include 32 bits (orequivalently 4 bytes). The size of the picture frame may be indicated inunits of bytes, but other units may be used. This allows theencoding/decoding process flexibility in encoding picture frames ofdifferent size (e.g., 1024×720 pixels, 2048×1540 pixels, etc.) and allowthe decoding module to determine the correct picture frame size to beused for a specific bytestream.

A decoding module would obtain said second field from the bytestream anddetermine, based on the value included in said second field, a size of apicture frame corresponding to said bytestream. The decoding module mayuse and/or implement a decoding process to adapt to said size, and inparticular to reconstruct the picture frame from the encoded bytestreamto fit into said size.

The fixed-sized header may include a third field indicating arecommended number of bits/bytes to fetch/retrieve at the decodingmodule when obtaining the bytestream (B.3—also described asrecommended_fetch_size: unit32). In an embodiment, this first field mayinclude 32 bits (or equivalently 4 bytes). This field may beparticularly useful in certain applications and/or for certain decodingmodules when retrieving the bytestream from a server, for example toenable the bytestream to be fetched/retrieved at the decoding module in“portions”. For example, this may enable partial decoding of thebytestream (as further described, for example, in European patentapplication No 17386047.9 filed on 6 Dec. 2017 by the same applicantwhose contents are included in their entirety by reference) and/oroptimise the retrieval of the bytestream by the decoding module (as forexample further described in European patent application No 12759221.0filed on 20 Jul. 2012 by the same applicant whose contents are includedin their entirety by reference).

A decoding module would obtain said third field from the bytestream anddetermine, based on the value included in said third field, a number ofbits and/or bytes of the bytestream to be retrieved from a separatemodule (for example, a server and/or a content delivery network). Thedecoding module may use and/or implement a decoding process to requestto the separate module said number of bits and/or bytes from thebytestream, and retrieve them from the separate module.

The fixed-sized header may include another field indicating a genericvalue in the bytestream (B.3.1—also described as element_interpretation:uint8). In an embodiment, this first field may include 8 bits (orequivalently 1 byte).

A decoding module would obtain said another field from the bytestreamand determine, based on the value included in said another field, avalue indicated by the field.

The fixed-sized header may include a fourth field indicating varioussystem information, including the type of transform operation to be usedin the decoding process (B.4—also described as pipeline: unit8). In anembodiment, this first field may include 8 bits (or equivalently 1byte). A transform operation is typically an operation that transform avalue from an initial domain to a transformed domain. One example ofsuch a transform is an integer composition transform. Another example ofsuch a transform is a composition transform. The composition transform(integer and/or standard) are further described in European patentapplication No. 13722424.2 filed on 13 May 2013 by the same applicantand incorporated herein by reference.

A decoding module would obtain said fourth field from the bytestream anddetermine, based on at least one value included in said fourth field, atype of transform operation to be used in the decoding process. Thedecoding module may configure the decoding process to use the indicatedtransform operation and/or implement a decoding process which uses theindicated transform operation when converting one or more decodedtransformed coefficient and/or value (e.g., a residual) into an originalnon-transform domain.

The fixed-sized header may include a fifth field indicating a type ofup-sampling filtering operation to be used in the decoding process(B.5—also described as upsampler: unit8). In an embodiment, this firstfield may include 8 bits (or equivalently 1 byte). An up-samplingfiltering operation comprises a filter which applies certainmathematical operations to a first number of samples/values to produce asecond number of samples/values, wherein the second number is higherthan the first number. The mathematical operations can either bepre-defined, adapted either based on an algorithm (e.g., using a neuralnetwork or some other adaptive filtering technique) or adapted based onadditional information received at the decoding module. Examples of suchup-sampling filtering operations comprise a Nearest Neighbour filteringoperation, a Sharp filtering operation, a Bi-cubic filtering operation,and a Convolutional Neural Network (CNN) filtering operations. Thesefiltering operations are described in further detail in the presentapplication, as well as in UK patent application No. 1720365.4 filed on6 Dec. 2017 by the same applicant and incorporated herein by reference.

A decoding module would obtain said fifth field from the bytestream anddetermine, based on at least one value included in said fifth field, atype of up-sampling operation to be used in the decoding process. Thedecoding module may configure the decoding process to use the indicatedup-sampling operation and/or implement a decoding process which uses theindicated up-sampling operation. The indication of the upsamplingoperation to be used allows flexibility in the encoding/decodingprocess, for example to better suit the type of picture to beencoded/decoded based on its characteristics.

The fixed-sized header may include a sixth field indicating one or moremodifying operations used in the encoding process when building thefixed-sized header and/or other headers and/or to be used in thedecoding process in order to decode the bytestream (see below) (B.6—alsodescribed as shortcuts: shortcuts_t). These modifying operations arealso called shortcuts. The general advantage provided by these shortcutsis to reduce the amount of data to be encoded/decoded and/or to optimisethe execution time at the decoder, for example by optimising theprocessing of the bytestream.

A decoding module would obtain said sixth field from the bytestream anddetermine, based on at least one value included in said sixth field, atype of shortcut used in the encoding process and/or to be used in thedecoding process. The decoding module may configure the decoding processto adapt its operations based on the indicated shortcut and/or implementa decoding process which uses the indicated shortcut.

The fixed-sized header may include a seventh field indicating a firstnumber of bits to be used to represent an integer number and a secondnumber of bits to be used to represent a fractional part of a number(B.7—also described as element_descriptor: tuple (uint5, utin3)). In anembodiment, this first field may include 8 bits (or equivalently 1 byte)subdivided in 5 bits for the first number of bits and 3 bits for thesecond number of bits.

A decoding module would obtain said seventh field from the bytestreamand determine, based on at least one value included in said seventhfield, how many bits to dedicate to represent the integer part of anumber that has both integer and fractional parts and how many bits todedicate to a fractional number.

The fixed-sized header may include an eighth field indicating a numberof planes forming a frame and to be used when decoding the bytestream(B.8—also described as num_plane: unit8). In an embodiment, this firstfield may include 8 bits (or equivalently 1 byte). A plane is defined inthe present application and is, for example, one of the dimensions in acolor space, for examples the luminance component Y in a YUV space, orthe red component R in an RGB space.

A decoding module would obtain said eighth field from the bytestream anddetermine, based on at least one value included in said fifth field, thenumber of planes included in a picture.

The fixed-sized header may include a ninth field indicating a size of anauxiliary header portion included in a separate header—for example theFirst Variable-Size Header or the Second Variable-Size Header (B.9—alsodescribed as aux_header_size: uunt16). In an embodiment, this firstfield may include 16 bits (or equivalently 2 byte). This field allowsthe encoding/decoding process to be flexible and define potentialadditional header fields.

A decoding module would obtain said ninth field from the bytestream anddetermine, based on at least one value included in said ninth field, asize of an auxiliary header portion included in a separate header. Thedecoding module may configure the decoding process to read the auxiliaryheader in the bytestream.

The fixed-sized header may include a tenth field indicating a number ofauxiliary attributes (B.10—also described as num_aux_tile_attribute:uint4 and num_aux_tableau_attribute: uint4). In an embodiment, thisfirst field may include 8 bits (or equivalently 1 byte) split into two4-bits sections. This field allows the encoding/decoding process to beflexible and define potential additional attributes for both Tiles andTableaux. These additional attributes may be defined in theencoding/decoding process.

A decoding module would obtain said tenth field from the bytestream anddetermine, based on at least one value included in said tenth field, anumber of auxiliary attributes associated with a tile and/or a number ofauxiliary attributes associated with a tableau. The decoding module mayconfigure the decoding process to read said auxiliary attributes in thebytestream.

The bytestream may include a first variable-sized header, i.e. a headerwhose byte/bit length is changeable depending on the data beingtransmitted within it. The header may include a plurality of fields.FIG. 21 shows an example of said first variable-sized header.

The first variable-sized header may include a first field indicating asize of a field associated with an auxiliary attribute of a tile and/ora tableau (C.1—also described as aux_attribute_sizes: until6[num_aux_the_attribute+num_aux_tableau_attribute]). In an embodiment,the second field may include a number of sub-fields, each indicating asize for a corresponding auxiliary attribute of a tile and/or a tableau.The number of these sub-fields, and correspondingly the number ofauxiliary attributes for a tile and/or a tableau, may be indicated in afield of a different header, for example the fixed header describedabove, in particular in field B.10. In an embodiment, this first fieldmay include 16 bits (or equivalently 2 bytes) for each of the auxiliaryattributes. Since the auxiliary attributes may not be included in thebytestream, this field would allow the encoding/decoding process todefine the size of the auxiliary attributes were they to be included inthe bytestream. This contrasts, for example, with the attributes (seefor example C.2 below) which typically are pre-defined in size andtherefore their size does not need to be specified and/or communicated.

A decoding module would obtain said first field from the bytestream anddetermine, based on a value included in said first field, a size of anauxiliary attribute associated with a tessera, (i.e., either a tile or atableau). In particular, the decoding module may obtain from said firstfield in the bytestream, a size of an auxiliary attribute for each ofthe auxiliary attributes which the decoding module is expecting todecode, for example based on information received separately about thenumber of auxiliary attributes to be specified. The decoding module mayconfigure the decoding process to read the auxiliary attributes in thebytestream.

The first variable-sized header may include a second field indicating,for each attribute of a tile and/or a tableau, a number of differentversions of the respective attribute (C.2—also described asnums_attribute:unti16[4+num_aux_tile_attribute+num_aux_tableau_attribute]). The secondfield may include a number of sub-fields, each indicating for acorresponding attribute a number of different version of said respectiveattribute. The number of these sub-fields, and correspondingly thenumber of standard attributes and auxiliary attributes for a tile and/ora tableau, may be indicated at least in part in a field of a differentheader, for example the fixed header described above, in particular infield B.10. The attributes may comprise both standard attributesassociated with a tile and/or a tableau and the auxiliary attributes asdescribed above. In an embodiment, there are three standard attributesassociated with a tile (e.g., Residual Statistics, T-Node Statistics andQuantization Parameters) and two standard attributes associated with atableau (e.g., Streamlengths Statistics and T-Node Statistics). In anembodiment, since the T-Node Statistics for the tiles and the tableauxmay be the same, they may only require to be specified once. In suchembodiment, only four different standard attributes will need to beincluded (and therefore only four sub-fields, C.2.1 to C.2.4, eachassociated with one of the four standard attributes Residual Statistics,T-Node Statistics, Quantization Parameters and Streamlengths Statistics,are included in the second field, each indicating a number of differentversions of the respective attribute). Accordingly, there may be fourdifferent sub-fields in said second field, each indicating the number ofstandard attributes for a tile and/or a tableau which need to bespecified for the decoding process. By way of example, if the sub-fieldassociated with the T-Node Statistics indicate a number 20, it meansthat there will be 20 different available versions of T-Node Statisticsto use for tiles and/or attributes.

A decoding module would obtain said second field from the bytestream anddetermine, based on a value included in said second field, a number ofdifferent versions of a respective attribute, said attribute associatedwith a tile and/or a tableau. The decoding module may configure thedecoding process to use the available versions of the correspondingattributes.

The first variable-sized header may include a third field indicating anumber of different groupings of tiles, wherein each grouping of tilesis associated with a common attribute (C.3—also described asnum_tileset: uint16). In an embodiment, this first field may include 16bits (or equivalently 2 bytes). In an embodiment, the common attributemay be the T-Node Statistics for a tile. For example, if a grouping oftiles (also known as “tileset”) is associated with the same T-nodeStatistics, it means that all the tiles in that grouping shall beassociated with the same T-Node Statistics. The use of grouping of tilessharing one or more common attributes allows the coding and decodingprocess to be flexible in terms of specifying multiple versions of asame attribute and associate them with the correct tiles. For example,if a group of tiles belongs to “Group A”, and “Group A” is associatedwith “Attribute A” (for example, a specific T-Node Statistics), then allthe tiles in Group A shall use that Attribute A. Similarly, if a groupof tiles belongs to “Group B”, and “Group B” is associated with“Attribute B” (for example, a specific T-Node Statistics different fromthat of Group A), then all the tiles in Group B shall use that AttributeB. This is particularly useful in allowing the tiles to be associatedwith a statistical distribution as close as possible to that of the tilebut without having to specify different statistics for every tile. Inthis way, a balance is reached between optimising the entropy encodingand decoding (optimal encoding and decoding would occur if thedistribution associated with the tile is the exact distribution of thattile) whilst minimising the amount of data to be transmitted. Tiles aregrouped, and a “common” statistics is used for that group of tiles whichis as close as possible to the statistics of the tiles included in thatgrouping. For example, if we have 256 tiles, in an ideal situation wewould need to send 256 different statistics, one for each of the tiles,in order to optimise the entropy encoding and decoding process (anentropy encoder/decoder is more efficient the more the statisticaldistribution of the encoded/decoded symbols is close to the actualdistribution of said symbols). However, sending statistics isimpractical and expensive in terms of compression efficiency. So,typical systems would send only one single statistics for all the 256tiles. However, if the tiles are grouped into a limited number ofgroupings, for example 10, with each tile in each grouping havingsimilar statistics, then only 10 statistics would need to be sent. Inthis way, a better encoding/decoding would be achieved than if only onecommon statistics was to be sent for all the 256 tiles, whilst at thesame time sending only 10 statistics and therefore not compromising toomuch the compression efficiency.

A decoding module would obtain said third field from the bytestream anddetermine, based on a value included in said third field, a number ofdifferent groupings of tiles. The decoding module may configure thedecoding process to use, when decoding a tile corresponding to aspecific grouping, one or more attributes associated with said grouping.

The first variable-sized header may include a fourth field indicating anumber of different groupings of tableaux, wherein each grouping oftableaus is associated with a common attribute (C.4—also described asnum_tableauset: uint16). In an embodiment, this fourth field may include16 bits (or equivalently 2 bytes). This field works and is based on thesame principles as the third field, except that in this case it refersto tableaux rather than tiles.

A decoding module would obtain said fourth field from the bytestream anddetermine, based on a value included in said fourth field, a number ofdifferent groupings of tableaux. The decoding module may configure thedecoding process to use, when decoding a tableau corresponding to aspecific grouping, one or more attributes associated with said grouping.

The first variable-sized header may include a fifth field indicating awidth for each of a plurality of planes (C.5—also described as widths:uint16[num_plane]). In an embodiment, this fifth field may include 16bits (or equivalently 2 bytes) for each of the plurality of planes. Aplane is further defined in the present specification, but in general isa grid (usually a two-dimensional one) of elements associated with aspecific characteristic, for example in the case of video thecharacteristics could be luminance, or a specific color (e.g. red, blueor green). The width may correspond to one of the dimensions of a plane.Typically, there are a plurality of planes.

A decoding module would obtain said fifth field from the bytestream anddetermine, based on a value included in said fifth field, a firstdimension associated with a plane of elements (e.g., picture elements,residuals, etc.). This first dimension may be the width of said plane.The decoding module may configure the decoding process to use, whendecoding the bytestream, said first dimension in relation to itsrespective plane.

The first variable-sized header may include a sixth field indicating awidth for each of a plurality of planes (C.6—also described as heights:uint16[num_plane]). In an embodiment, this sixth field may include 16bits (or equivalently 2 bytes) for each of the plurality of planes. Theheight may correspond to one of the dimensions of a plane.

A decoding module would obtain said sixth field from the bytestream anddetermine, based on a value included in said sixth field, a seconddimension associated with a plane of elements (e.g., picture elements,residuals, etc.). This second dimension may be the height of said plane.The decoding module may configure the decoding process to use, whendecoding the bytestream, said second dimension in relation to itsrespective plane.

The first variable-sized header may include a seventh field indicating anumber of encoding/decoding levels for each of a plurality of planes(C.7—also described as num_loqs: uint8[num_plane]). In an embodiment,this seventh field may include 16 bits (or equivalently 2 bytes) foreach of the plurality of planes. The encoding/decoding levels correspondto different levels (e.g., different resolutions) within a hierarchicalencoding process. The encoding/decoding levels are also referred in theapplication as Level of Quality

A decoding module would obtain said seventh field from the bytestreamand determine, based on a value included in said seventh field, a numberof encoding levels for each of a plurality of planes (e.g., pictureelements, residuals, etc.). The decoding module may configure thedecoding process to use, when decoding the bytestream, said number ofencoding levels in relation to its respective plane.

The first variable-sized header may include an eighth field containinginformation about the auxiliary attributes (C.8—also described asaux_header: uint8[aux_header_size]). In an embodiment, this eight fieldmay include a plurality of 8 bits (or equivalently 1 byte) depending ona size specified, for example, in a field of the fixed header (e.g.,B.9)

A decoding module would obtain said eighth field from the bytestream anddetermine information about the auxiliary attributes. The decodingmodule may configure the decoding process to use, when decoding thebytestream, said information to decode the auxiliary attributes.

The bytestream may include a second variable-sized header, i.e. a headerwhose byte/bit length is changeable depending on the data beingtransmitted within it. The header may include a plurality of fields.FIG. 22 shows an example of said second variable-sized header.

The second variable-sized header may include a first field containing,for each attribute, information about one or more statistics associatedwith the respective attribute (see D.1). The number of statisticsassociated with a respective attribute may be derived separately, forexample via field C.2 as described above. The statistics may be providedin any form. In an embodiment of the present application, the statisticsis provided using a particular set of data information which includesinformation about a cumulative distribution function (typeresidual_stat_t).

In particular, a first group of sub-fields in said first field maycontain information about one or more statistics associated withresiduals values (also D.1.1—also described as residual_stats:residual_stat_t[nums_attribute[0]]). In other words, the statistics mayidentify how a set of residual data are distributed. The number ofstatistics included in this first group of sub-fields may be indicatedin a separate field, for example in the first sub-field C.2.1 of fieldC.2 as described above (also indicated as nums_attribute[0]). Forexample, if nums_attribute[0] is equal to 10, then there would be 10different residuals statistics contained in said first field. Forexample, the first 10 sub-fields in the first field correspond to saiddifferent 10 residuals statistics.

A second group of sub-fields in said first field may contain informationabout one or more statistics associated with nodes within a Tessera(also D.1.2—also described as tnode_stats:tnode_stat_t[nums_attribute[1]]). In other words, the statistics mayidentify how a set of nodes are distributed. The number of statisticsincluded in this second group of sub-fields may be indicated in aseparate field, for example in the second sub-field C.2.2 of field C.2as described above (also indicated as nums_attribute[1]). For example,if nums_attribute[1] is equal to 5, then there would be 5 differentt-node statistics contained in said first field. For example,considering the example above, after the first 10 sub-fields in thefirst field, the next 5 sub-fields correspond to said 5 different t-nodestatistics.

A third group of sub-fields in said first field may contain informationabout one or more quantization parameters (also D.1.3—also described asquantization parameters: quantization_parameters_t[nums_attribute[2]]).The number of quantization parameters included in this third group ofsub-fields may be indicated in a separate field, for example in thethird sub-field C.2.3 of field C.2 as described above (also indicated asnums_attribute[2]). For example, if nums_attribute[2] is equal to 10,then there would be 10 different quantization parameters contained insaid first field. For example, considering the example above, after thefirst 15 sub-fields in the first field, the next 10 sub-fieldscorrespond to said 10 different quantization parameters.

A fourth group of sub-fields in said first field may contain informationabout one or more statistics associated with streamlengths (alsoD.1.4—also described as stream_length_stats:stream_length_stat_t[nums_attribute[3]]). In other words, the statisticsmay identify how a set of streamlengths are distributed. The number ofstatistics included in this fourth group of sub-fields may be indicatedin a separate field, for example in the fourth sub-field C.2.4 of fieldC.2 as described above (also indicated as nums_attribute[3]). Forexample, if nums_attribute[4] is equal to 12, then there would be 12different streamlengths statistics contained in said first field. Forexample, considering the example above, after the first 25 sub-fields inthe first field, the next 12 sub-fields correspond to said 12 differentstreamlengths statistics.

Further groups of sub-fields in said first field may contain informationabout auxiliary attributes (also described as aux_atttributes:uint1[aux_attributes_size[i]][num_aux_the_attribute+num_aux_tableau_attribute]). The number ofauxiliary attributes may be indicated in another field, for example infield C.2 as described above.

Specifying one or more versions of the attributes (e.g., statistics)enables flexibility and accuracy in the encoding and decoding process,because for instance more accurate statistics can be specified for aspecific grouping of tesserae (tiles and/or tableaux), thus making itpossible to encode and/or decode said groupings in a more efficientmanner.

A decoding module would obtain said first field from the bytestream anddetermine, based on the information contained in said first field, oneor more attributes to be used during the decoding process. The decodingmodule may store the decoded one or more attributes for use during thedecoding process. The decoding module may, when decoding a set of data(for example, a tile and/or a tableau) and based on an indication ofattributes to use in relation to that set of data, retrieve theindicated attributes from the stored decoded one or more attributes anduse it in decoding said set of data.

The second variable-sized header may include a second field containing,for each of a plurality of grouping of tiles, an indication of acorresponding set of attributes to use when decoding said grouping(D.2—also described as tilesets: uint16[3+num_aux_tile_attributes][num_tiles]). The number of groupings of tiles may be indicated in aseparate field, for example in field C.3 described above. This secondfield enables the encoding/decoding process to specify which of the setsof attributes indicated in field D.1 described above is to be used whendecoding a tile.

A decoding module would obtain said second field from the bytestream anddetermine, based on the information contained in said second field,which of a set of attributes is to be used when decoding a respectivegrouping of tiles. The decoding module would retrieve from a repositorystoring all the attributes the ones indicated in said second field, anduse them when decoding the respective grouping of tiles. The decodingprocess would repeat said operations when decoding each of the pluralityof grouping of tiles.

By way of example, and using the example described above in relation tofield D.1, let's assume that for a first grouping of tiles the set ofattributes indicated in said second field corresponds to residualsstatistics No. 2, t_node statistics No. 1 and to quantization parameterNo. 4 (we assume for simplicity that there are no auxiliary attributes).When the receiving module receives said indication, it would retrievefrom the stored attributes (as described above) the second residualsstatistics from the 10 stored residuals statistics, the first t_nodestatistics from the 5 stored t_node statistics and the fourthquantization parameter from the 10 stored quantization parameters.

The second variable-sized header may include a fourth field containing,for each of a plurality of grouping of tableaux, an indication of acorresponding set of attributes to use when decoding said grouping(D.4—also described as tableausets:uint16[2+num_aux_tableaux_attributes] [num_tableaux]). The number ofgroupings of tableaux may be indicated in a separate field, for examplein field C.4 described above. This fourth field enables theencoding/decoding process to specify which of the sets of attributesindicated in field D.1 described above is to be used when decoding atableau.

The principles and operations behind this fourth field corresponds tothat described for the second field, with the difference that in thiscase it applies to tableaux rather than tiles. In particular, a decodingmodule would obtain said fourth field from the bytestream and determine,based on the information contained in said fourth field, which of a setof attributes is to be used when decoding a respective grouping oftableaux. The decoding module would retrieve from a repository storingall the attributes the ones indicated in said fourth field, and use themwhen decoding the respective grouping of tableaux. The decoding processwould repeat said operations when decoding each of the plurality ofgrouping of tableaux.

The second variable-sized header may include a fifth field containing,for each plane, each encoding/decoding level and each direction, anindication of a corresponding set of attributes to use when decoding aroot tableau (D.5—also described as root_tableauset_indices:uint16[loq_idx][num_planes][4]). This fifth field enables theencoding/decoding process to specify which of the sets of attributesindicated in field D.1 described above is to be used when decoding aroot tableau.

A decoding module would obtain said fifth field from the bytestream anddetermine, based on the information contained in said fifth field, whichof a set of attributes is to be used when decoding a respective roottableau. The decoding module would retrieve from a repository storingall the attributes the ones indicated in said fifth field, and use themwhen decoding the respective grouping of tiles.

In this way, the decoding module would effectively store all thepossible attributes to be used when decoding tiles and/or tableauxassociated with that bytestream, and then retrieve for each of agrouping of tiles and/or tableaux only the sub-set of attributesindicated in said second field to decode the respective grouping oftiles and/or tableaux.

The second variable-sized header may include a third field containinginformation about the statistics of the groupings of tiles (D.3—alsodescribed as cdf_tilesets: line_segments_cdf15_t<tilese_index_t>). Thestatistics may provide information about how many times a certaingrouping of tiles occurs. The statistics may be provided in the form ofa cumulative distribution function. In the present application, the waythe cumulative distribution function is provided is identified as afunction type, specifically type line_segments_cdf15_t<x_axis_type>. Byusing said statistics, the encoding/decoding process is enabled tocompress the information about the grouping of tiles (e.g., the indicesof tiles) and therefore optimise the process. For example, if there areN different groupings of tiles, and correspondingly N different indexes,rather than transmitting these indexes in an uncompressed manner, whichwould require 2^(┌ log) ² ^(N┐) bits (where ┌.┐ is a ceiling function),the grouping can be compressed using an entropy encoder thus reducingsignificantly the number of bits required to communicate the groupingsof tiles. This may represent a significant savings. For example, assumethat there are 10,000 tiles encoded in the bytestream, and that thesetiles are divided in 100 groupings. Without compressing the indexes, anindex needs to be sent together with each tile, meaning that at least2^(┌ log) ² ^(100┐)=7 bits per tile, which means a total of 70,000 bits.If instead the indexes are compressed using an entropy encoder to anaverage of 1.5 bits per index, the total number of bits to be used wouldbe 15,000, reducing the number of bits to be used by almost 80%.

A decoding module would obtain said third field from the bytestream anddetermine, based on the information contained in said third field,statistical information about the groupings of tiles. The decodingmodule would use said statistical information when delving whichgrouping a tile belongs to. For example, the information about the tilegrouping (e.g., tileset index) can be compressed using said statisticsand then reconstructed at the decoder using the same statistics, forexample using an entropy decoder.

The second variable-sized header may include a sixth field containinginformation about the statistics of the groupings of tableaux (D.6—alsodescribed as cdf_tableausets:line_segments_cdf15_t<tableauset_index_t>). The statistics may provideinformation about how many times a certain grouping of tableaux occurs.The statistics may be provided in the form of a cumulative distributionfunction.

This field works in exactly the same manner as the third field but forgrouping of tableaux rather than grouping of tiles. In particular, adecoding module would obtain said sixth field from the bytestream anddetermine, based on the information contained in said sixth field,statistical information about the groupings of tableaux. The decodingmodule would use said statistical information when deriving whichgrouping a tableau belongs to. For example, the information about thetableau grouping (e.g., tableauset index) can be compressed using saidstatistics and then reconstructed at the decoder using the samestatistics, for example using an entropy decoder.

The second variable-sized header may include a seventh field containing,for each plane, each encoding/decoding level and each direction, anindication of a location, within a payload of the bytestream, of one ormore sub-streams (e.g., a Surface) of bytes associated for thatrespective plane, encoding/decoding level and direction (D.7—alsodescribed as root_stream_offsets:root_stream_offset_t[loq_idx][num_planes][4]). The location may beindicated as an offset with respect to the start of the payload. By wayof example, assuming 3 planes, 3 encoding/decoding levels and 4directions, there will be 3*3*4=36 different sub-streams, andcorrespondingly there will be 36 different indication of locations(e.g., offsets).

A decoding module would obtain said seventh field from the bytestreamand determine, based on the information contained in said seventh field,where to find in the payload a specific sub-stream. The sub-stream maybe associated with a specific direction contained in a specific planewhich is within a specific encoding/decoding level. The decoding modulewould use said information to locate the sub-stream and decode saidsub-stream accordingly. The decoding module may implement, based on thisinformation, decoding of the various sub-stream simultaneously and/or inparallel. This can be advantageous for at least two reasons. First, itwould allow flexibility in ordering of the sub-streams. The decodercould reconstruct, based on the location of the sub-streams, to whichdirection, plane and encoding/decoding level the sub-stream belongs to,without the need for that order to be fixed. Second, it would enable thedecoder to decode the sub-streams independently from one another aseffectively each sub-stream is separate from the others.

The second variable-sized header may include an eighth field containing,for each plane, each encoding/decoding level and each direction, a sizeof the Stream of bytes associated with the root tableau (D.8—alsodescribed as root_stream_lengths:root_stream_length_t[loq_idx][num_planes][4]).

A decoding module would obtain said eighth field from the bytestream anddetermine, based on the information contained in said eighth field, thelength of a stream associated with a root tableau.

Further numbered statements of examples described in this documentinclude the following statements.

1. A method of processing metadata associated with a stream of videodata, the method comprising the steps of, for a first frame of videodata:

-   -   subdividing the first frame of video data into a plurality of        tiles;    -   calculating a first statistical attribute for each of a        plurality of tiles;    -   identifying tiles having a first instance of identical, or        similar statistical, attributes, and grouping said identified        tiles together as a tile set;    -   for each tile set defining metadata for the tile set, said        metadata indicative of the statistical attribute of the tiles        defining the tile set; and    -   encoding data indicative of the metadata for the tiles of the        first frame of video data based on the metadata defined for each        of the tile set to which said tile belongs.

2. The method of statement 1 wherein the encoding occurs using anentropy encoding based technique.

3. The method of any preceding statement wherein the metadata for a tileset further defines a location of each of the tiles forming the tileset.

4. The method of any preceding statement wherein the size of the tilesis fixed.

5. The method of any preceding statement wherein the step of identifyingthe tiles that form a tile set further comprises ordering the tiles,based on their statistical attributes.

6. The method of statement 5 wherein the ordering of the tiles defines aprobability distribution function of the statistical attributes.

7. The method of any preceding statement wherein tiles are encoded in aset order.

8. The method of statement 7 wherein the method further comprisesdetermining the difference between the metadata of a tile and itspreceding tile, and encoding the metadata as the difference.

9. The method of statement 7 or 8 wherein in the set order is a z-ordertraverse.

10. The method of any preceding statement wherein the method furthercomprise the step of encoding the tiles.

11. The method of statement 10 wherein the step of encoding the tilescomprises determining a level of quantisation and encoding tiles at saidlevel of quantisation

12. The method of statement 11 wherein tiles belonging to a first afirst tile set are quantised at a first level of quantisation and tilesbelonging to a second tile set are quantised at a second, different,level of quantisation.

13. The method of any preceding statement wherein the statisticalattributes of the tile are selected from one or more of the group of:luma, chroma, and number of bits required to encode one or more pixels,within a frame of video data.

14. The method of any preceding statement wherein the first frame ofvideo data is a residual frame, said residual frame being indicative ofthe differences between a first frame of data and a reference frame.

15. The method of any preceding statement wherein the method furthercomprises identifying one or more further statistical attributes of thetiles and identifying tiles having a plurality of instances ofidentical, or similar statistical, attributes, and grouping saididentified tiles together as the tile set.

16. A system for encoding metadata associated with a stream of videodata, the system comprising a processor, the processor configured to,for a first frame of video data:

-   -   subdivide the first frame of video data into a plurality of        tiles;    -   for each of a plurality of tiles calculate a first statistical        attributes of the tile;    -   identify tiles having a first instance of identical, or similar        statistical, attributes, and group said identified tiles        together as a tile set;    -   for each tile set define metadata for the tile set, said        metadata indicative of the statistical attribute of the tiles        defining the tile set; and    -   encode data indicative of the metadata for the tiles of the        first frame of video data based on the metadata defined for each        of the tile set to which said tile belongs.

17. A method of decoding metadata associated with a stream of videodata, the method comprising the steps of, for a first frame of videodata, at a decoder:

-   -   obtaining information to enable the decoder to subdivide the        first frame of video data into a plurality of tiles;    -   receiving an encoded stream of metadata, said encoded stream of        metadata comprising information identifying tiles having a first        instance of identical, or similar statistical, attributes, and        information grouping said identified tiles together as a tile        set;    -   obtaining information regarding the identical, or similar        statistical, attributes; and    -   decoding the metadata for each of the tiles forming the tile set        with the obtained information regarding the identical, or        similar statistical, attributes.

18. The method of statement 17 wherein the encoded data stream isdecoded using an entropy encoding based technique.

19. The method of statement 16 or 17 wherein the decoded metadata for atile set further defines a location of each of the tiles forming thetile set.

20. The method of any of statements 16 to 19 wherein the size of thetiles is fixed.

21. The method of any of statements 16 to 20 comprising the step ofobtaining information regarding the order in which the tiles wereencoded and decoding the encoded stream based on said order.

22. The method of statement 22 wherein the method further comprisesdetermining the difference between the metadata of a tile and itspreceding tile, and decoding the metadata as the difference.

23. The method of statement 21 or 22 wherein in the set order is az-order traverse.

24. The method of any of statements 16 to 23 further comprisingobtaining information regarding a level of quantisation and decoding thedata stream at said level of quantisation

25. The method of statement 24 wherein tiles belonging to a first afirst tile set are decoded at a first level of quantisation and tilesbelonging to a second tile set are decoded at a second, different, levelof quantisation.

26. A decoder for decoding an encoded stream of video data, the decoderconfigured to perform the method of any of statements 17 to 25.

A-1. A method of decoding metadata associated with a stream of videodata, the method comprising the steps of, for a first frame of videodata, at a decoder:

-   -   obtaining information to enable the decoder to subdivide the        first frame of video data into a plurality of tiles;    -   receiving an encoded stream of metadata, said encoded stream of        metadata comprising information identifying tiles having a first        instance of identical, or similar statistical, attributes, and        information grouping said identified tiles together as a tile        set;    -   obtaining information regarding the identical, or similar        statistical, attributes; and    -   decoding the metadata for each of the tiles forming the tile set        with the obtained information regarding the identical, or        similar statistical, attributes.

A-2. The method of statement A-1 wherein the encoded data stream isdecoded using an entropy encoding based technique.

A-3. The method of statement A-1 or A-2 wherein the decoded metadata fora tile set further defines a location of each of the tiles forming thetile set.

A-4. The method of any preceding statement wherein the size of the tilesis fixed.

A-5. The method of any preceding statement comprising the step ofobtaining information regarding the order in which the tiles wereencoded and decoding the encoded stream based on said order.

A-6. The method of statement A-5 wherein the method further comprisesdetermining the difference between the metadata of a tile and itspreceding tile, and decoding the metadata as the difference.

A-7. The method of statement A-5 or A-6 wherein in the set order is az-order traverse.

A-8. The method of any preceding statement further comprising obtaininginformation regarding a level of quantisation and decoding the datastream at said level of quantisation

A-9. The method of statement A-8 wherein tiles belonging to a first afirst tile set are decoded at a first level of quantisation and tilesbelonging to a second tile set are decoded at a second, different, levelof quantisation.

A-10 A decoder for decoding an encoded stream of video data, the decoderconfigured to perform the method of any of statements A-1 to A-9.

A-11. A method of processing metadata associated with a stream of videodata, the method comprising the steps of, for a first frame of videodata:

-   -   subdividing the first frame of video data into a plurality of        tiles;    -   calculating a first statistical attribute for each of a        plurality of tiles;    -   identifying tiles having a first instance of identical, or        similar statistical, attributes, and grouping said identified        tiles together as a tile set;    -   for each tile set defining metadata for the tile set, said        metadata indicative of the statistical attribute of the tiles        defining the tile set; and    -   encoding data indicative of the metadata for the tiles of the        first frame of video data based on the metadata defined for each        of the tile set to which said tile belongs.

A-12. The method of statement A-11 wherein the encoding occurs using anentropy encoding based technique.

A-13. The method of any of statements A-11 or A-12 wherein the metadatafor a tile set further defines a location of each of the tiles formingthe tile set.

A-14. The method of statements A-11 to A-13 wherein the size of thetiles is fixed.

A-15. The method of statements A-11 to A-14 wherein the step ofidentifying the tiles that form a tile set further comprises orderingthe tiles, based on their statistical attributes.

1. A method of decoding a received set of encoded data representinginformation that has been compressed, the method comprising: obtaining,a set of attribute indicators from the data set, each indicator of theset of indicators is associated with a subset of the data set; and,decoding a plurality of subsets of the data set, comprising: retrievingdecoding parameters for each subset according to the attribute indicatorassociated with each subset; and, decoding each subset according to theretrieved decoding parameters for each subset.
 2. A method according toclaim 1, wherein the step of retrieving comprises retrieving a pluralityof decoding parameters according to each attribute indicator, theplurality of decoding parameters corresponding to different functions ofa decoding process.
 3. A method according to claim 1 or 2, wherein theplurality of subsets contain encoded data values and wherein the methodfurther comprises: recreating the information that has been compressedfrom the data values.
 4. A method according to any preceding claim,wherein each indicator of the set of indicators is associated with arespective subset of the plurality of subsets of the data set.
 5. Amethod according to any preceding claim, wherein a plurality of theindicators are identical.
 6. A method according to any preceding claim,wherein the encoded data set is divided into a hierarchy of subsets
 7. Amethod according to claim 6, wherein the attribute indicators of subsetscontaining data values are obtained by separately decoding an attributemetadata subset of the data set.
 8. A method according to any precedingclaim, further comprising: obtaining an initial attribute indicatorassociated with an initial subset of the data set; retrieving initialdecoding parameters of the initial subset according to the initialattribute indicator; decoding the initial subset according to theinitial decoding parameters, wherein the initial subset contains theattribute indicators of the plurality of subsets of the data set.
 9. Amethod according to any preceding claim, wherein the decoding parametersinclude quantization parameters.
 10. A method according to any precedingclaim, wherein the decoding parameters include entropy decodingprobability metadata.
 11. A method according to claim 10, wherein theentropy decoding probability metadata represents a cumulativedistribution function for a range decoding operation.
 12. An apparatusfor decoding an received set of encoded data representing informationthat has been compressed, comprising a processor configured to carry outthe method of any of claims 1 to
 11. 13. A method of encodinginformation to be compressed into an encoded data set, the methodcomprising: encoding a plurality of subsets of the data set, comprising:retrieving encoding parameters for each subset; and, encoding eachsubset according to the retrieved encoding parameters for each subset;and, generating a set of attribute indicators, each indicator of the setof indicators being associated with a subset of the data set accordingto the retrieved encoding parameters for each subset.
 14. A methodaccording to claim 13, wherein the step of retrieving comprisesretrieving a plurality of encoding parameters, the plurality of decodingparameters corresponding to different functions of an encoding process.15. A method according to claim 13 or 14, wherein the plurality ofsubsets contain encoded data values, such that the information to becompressed can be recreated from the data values.
 16. A method accordingto any of claims 13 to 15, wherein each indicator of the set ofindicators is associated with a respective subset of the plurality ofsubsets of the data set.
 17. A method according to any of claims 13 to16, wherein a plurality of the indicators are identical.
 18. A methodaccording to any of claims 13 to 17, wherein the encoded data set isdivided into a hierarchy of subsets
 19. A method according to claim 18,wherein the attribute indicators for subsets containing data values areseparately encoded as an attribute metadata subset of the data set. 20.A method according to any of claims 13 to 19, further comprising:encoding an initial subset according to retrieved initial decodingparameters, generating an initial attribute indicator associated with aninitial subset of the data set; wherein the encoded initial subsetcontains the attribute indicators of the plurality of subsets of thedata set.
 21. A method according to any of claims 13 to 20, wherein theencoding parameters include quantization parameters.
 22. A methodaccording to any of claims 13 to 21, wherein the encoding parametersinclude entropy coding probability metadata.
 23. A method according toclaim 22, wherein the entropy coding probability metadata represents acumulative distribution function for a range coding operation.
 24. Anapparatus for encoding information to be compressed into an encoded dataset, comprising a processor configured to carry out the method of any ofclaims 13 to
 23. 25. A method of processing metadata associated with astream of video data, the method comprising the steps of, for a firstframe of video data: subdividing the first frame of video data into aplurality of tiles; calculating a first statistical attribute for eachof a plurality of tiles; identifying tiles having a first instance ofidentical, or similar statistical, attributes, and grouping saididentified tiles together as a tile set; for each tile set definingmetadata for the tile set, said metadata indicative of the statisticalattribute of the tiles defining the tile set; and encoding dataindicative of the metadata for the tiles of the first frame of videodata based on the metadata defined for each of the tile set to whichsaid tile belongs.
 26. A method according to claim 25 wherein theencoding occurs using an entropy encoding based technique.
 27. A methodaccording to any of claims 25 to 26 wherein the metadata for a tile setfurther defines a location of each of the tiles forming the tile set.28. A method according to any of claims 25 to 27 wherein the size of thetiles is fixed.
 29. A method according to any of claims 25 to 28,wherein the step of identifying the tiles that form a tile set furthercomprises ordering the tiles, based on their statistical attributes. 30.A method according to claim 29 wherein the ordering of the tiles definesa probability distribution function of the statistical attributes.
 31. Amethod according to any of claims 25 to 30 wherein tiles are encoded ina set order.
 32. A method according to claim 31 wherein the methodfurther comprises determining the difference between the metadata of atile and its preceding tile, and encoding the metadata as thedifference.
 33. A method according to claim 31 or 32 wherein in the setorder is a z-order traverse.
 34. A method according to any of claims 25to 33 wherein the method further comprises the step of encoding thetiles.
 35. A method according to claim 34 wherein the step of encodingthe tiles comprises determining a level of quantisation and encodingtiles at said level of quantisation
 36. A method according to claim 35wherein tiles belonging to a first a first tile set are quantised at afirst level of quantisation and tiles belonging to a second tile set arequantised at a second, different, level of quantisation.
 37. A methodaccording to any of claim 25 to 36 wherein the statistical attributes ofthe tile are selected from one or more of the group of: luma, chroma,and number of bits required to encode one or more pixels, within a frameof video data.
 38. A method according to any of claims 25 to 37 whereinthe first frame of video data is a residual frame, said residual framebeing indicative of the differences between a first frame of data and areference frame.
 39. A method according to any of claims 25 to 38wherein the method further comprises identifying one or more furtherstatistical attributes of the tiles and identifying tiles having aplurality of instances of identical, or similar statistical, attributes,and grouping said identified tiles together as the tile set.
 40. Asystem for encoding metadata associated with a stream of video data, thesystem comprising a processor, the processor configured to, for a firstframe of video data: subdivide the first frame of video data into aplurality of tiles; for each of a plurality of tiles calculate a firststatistical attributes of the tile; identify tiles having a firstinstance of identical, or similar statistical, attributes, and groupsaid identified tiles together as a tile set; for each tile set definemetadata for the tile set, said metadata indicative of the statisticalattribute of the tiles defining the tile set; and encode data indicativeof the metadata for the tiles of the first frame of video data based onthe metadata defined for each of the tile set to which said tilebelongs.
 41. A method of decoding metadata associated with a stream ofvideo data, the method comprising the steps of, for a first frame ofvideo data, at a decoder: obtaining information to enable the decoder tosubdivide the first frame of video data into a plurality of tiles;receiving an encoded stream of metadata, said encoded stream of metadatacomprising information identifying tiles having a first instance ofidentical, or similar statistical, attributes, and information groupingsaid identified tiles together as a tile set; obtaining informationregarding the identical, or similar statistical, attributes; anddecoding the metadata for each of the tiles forming the tile set withthe obtained information regarding the identical, or similarstatistical, attributes.
 42. A method according to claim 41 wherein theencoded data stream is decoded using an entropy encoding basedtechnique.
 43. A method according to claim 40 or 41 wherein the decodedmetadata for a tile set further defines a location of each of the tilesforming the tile set.
 44. A method according to any of claims 41 to 43wherein the size of the tiles is fixed.
 45. A method according to any ofclaims 40 to 44 comprising the step of obtaining information regardingthe order in which the tiles were encoded and decoding the encodedstream based on said order.
 46. A method according to claim 45 whereinthe method further comprises determining the difference between themetadata of a tile and its preceding tile, and decoding the metadata asthe difference.
 47. A method according to claim 45 or 46 wherein in theset order is a z-order traverse.
 48. A method according to any of claims40 to 47 further comprising obtaining information regarding a level ofquantisation and decoding the data stream at said level of quantisation49. A method according to claim 48 wherein tiles belonging to a first afirst tile set are decoded at a first level of quantisation and tilesbelonging to a second tile set are decoded at a second, different, levelof quantisation.
 50. A decoder for decoding an encoded stream of videodata, the decoder configured to perform the method of any of claims 41to
 49. 51. A method of decoding an encoded data set comprising a headerand a payload, the payload comprising a plurality of encoded datasymbols representing information that has been compressed, wherein adecoder is configured to decode subsets of the payload according to apredetermined decoding process, the method comprising: decoding theheader to derive a set of metadata elements; retrieving from the set ofmetadata elements, a variation parameter which indicates how thepredetermined decoding process should be varied; decoding the payloadaccording to a modified decoding process based on the variationparameter, such that the decoding process is optimised for the pluralityof encoded data symbols; and, reconstructing the information that hasbeen compressed from the decoded payload.
 52. A method according toclaim 51, wherein the variation parameter indicates multiple subsets ofthe payload shall be decoded according one or more of the same decodingparameters.
 53. A method according to claim 51 or 52, wherein thevariation parameter indicates quantization is disabled, such that theplurality of encoded data symbols are not quantized during decoding. 54.A method according to any of claims 51 to 53, wherein the payloadcomprises a plurality of data structures each comprising metadata and aplurality of encoded data symbols, wherein the metadata indicatesfeatures of the data structures, encoded data symbols or both, andwherein the variation parameter indicates that expected optimisations onthe metadata of the optimisation process have not been performed.
 55. Amethod according to any of claims 51 to 54, wherein the predetermineddecoding process expects the payload to comprise a plurality of datastructures each comprising metadata and a plurality of encoded datasymbols, wherein the metadata indicates features of the data structure,encoded data symbols or both, and wherein the variation parameterindicates the data structures comprise only data symbols and do notcomprise metadata.
 56. A method according to any of claims 51 to 55,wherein the variation parameter indicates groups of subsets of thepayload shall be decoded according to the same decoding parameters. 57.A method according to any of claims 51 to 56, wherein the variationparameter indicates an expected decoding parameter for the payload issignalled explicitly in the encoded data set and is not signalled byreference.
 58. A method according to any of claims 51 to 57, wherein thepayload comprises a plurality of subsets, each subset being decodedaccording to a set of decoding parameters, and wherein the variationparameter indicates that an indicator element of the encoded data setthat indicates which set of decoding parameters shall be used for asubset references a list of tuples of a plurality of lists of tuples,each element of the tuple referencing a parameter to be used from adifferent set of parameters.
 59. A method according to any of claims 51to 58, wherein the payload comprises a plurality of subsets, each subsetbeing decoded according to a set of decoding parameters, and wherein thevariation parameter indicates that an indicator element of the encodeddata set that indicates which set of decoding parameters shall be usedfor a subset is a reference to a set of parameters from a plurality ofdifferent sets of parameters.
 60. An apparatus for decoding an encodeddata set comprising a header and a payload, comprising a processorconfigured to carry out the method of any of claims 51 to
 59. 61. Amethod of encoding a data set into an encoded data set comprising aheader and a payload, the payload comprising a plurality of encoded datasymbols representing information to be compressed, wherein a decoder isconfigured to decode subsets of the payload according to a predetermineddecoding process, the method comprising: retrieving a variationparameter which indicates how the predetermined decoding process shouldbe varied; encoding the variation parameter in a set of metadataelements into the header; encoding the information to be compressed intothe payload according to an encoding process based on the variationparameter, such that the decoding process is optimised for the pluralityof encoded data symbols.
 62. A method according to claim 61, wherein thevariation parameter indicates multiple subsets of the payload shall bedecoded according one or more of the same decoding parameters.
 63. Amethod according to claim 61 or 62, wherein the variation parameterindicates quantization is disabled, such that the plurality of encodeddata symbols are not quantized during decoding.
 64. A method accordingto any of claims 61 to 63, wherein the encoded payload comprises aplurality of data structures each comprising metadata and a plurality ofencoded data symbols, wherein the metadata indicates features of thedata structures, encoded data symbols or both, and wherein the variationparameter indicates that expected optimisations on the metadata of theoptimisation process have not been performed.
 65. A method according toany of claims 61 to 64, wherein the predetermined encoding processexpects the payload to comprise a plurality of data structures eachcomprising metadata and a plurality of encoded data symbols, wherein themetadata indicates features of the data structure, encoded data symbolsor both, and wherein the variation parameter indicates the encoded datastructures comprise only data symbols and do not comprise metadata. 66.A method according to any of claims 61 to 65, wherein the variationparameter indicates groups of subsets of the payload are encodedaccording to the same encoding parameters.
 67. A method according to anyof claims 61 to 66, wherein the variation parameter indicates anencoding parameter for the payload is signalled explicitly in theencoded data set and is not signalled by reference.
 68. A methodaccording to any of claims 61 to 67, wherein the payload comprises aplurality of subsets, each subset being encoded according to a set ofencoding parameters, and wherein the variation parameter indicates thatan indicator element of the encoded data set that indicates which set ofencoding parameters have been used for a subset references a list oftuples of a plurality of lists of tuples, each element of the tuplereferencing a parameter used from a different set of parameters.
 69. Amethod according to any of claims 61 to 68, wherein the payloadcomprises a plurality of subsets, each subset being encoded according toa set of encoding parameters, and wherein the variation parameterindicates that an indicator element of the encoded data set thatindicates which set of encoding parameters have been used for a subsetis a reference to a set of parameters from a plurality of different setsof parameters.
 70. An apparatus for encoding a data set into an encodeddata set comprising a header and a payload, the payload comprising aplurality of encoded data symbols representing information to becompressed, wherein a decoder is configured to decode subsets of thepayload according to a predetermined decoding process, the apparatuscomprising a processor configured to carry out the method of any ofclaims 61 to
 70. 71. A computer readable medium comprising instructionswhich cause a processor to carry out the method of any of claims 1 to11, 13 to 23, 25 to 39, 41 to 49, 51 to 59 and 61 to 69.